DOCUMENT RESUME 



ED 425 315 



CE 077 631 



AUTHOR 

TITLE 



INSTITUTION 
SPONS AGENCY 

PUB DATE 
NOTE 

CONTRACT 
AVAILABLE FROM 

PUB TYPE 
EDRS PRICE 
DESCRIPTORS 



IDENTIFIERS 



Jurmo, Paul 

Collaborative Learning for Continuous Improvement: Team 
Learning and Problem Solving in a Workplace Education 
Program. A Guidebook from a Project of the National 
Workplace Literacy Program. 

New York State Education Dept., Albany. 

Office of Vocational and Adult Education (ED) , Washington, 
DC. National Workplace Literacy Program. 

1998-08-00 

6 7p . ; For a related evaluation report, see ED 415 425. 
V198A40318 

Learning Partnerships, 14 Griffin Street, East Brunswick, NJ 
08816-4806; Tel: 732-254-2237. 

Guides - Non-Classroom (055) 

MF01/PC03 Plus Postage. 

Adult Basic Education; *Cooperative Learning; Educational 
Planning; Evaluation Methods; Inplant Programs; Literacy 
Education; Models; Needs Assessment; *Problem Solving; 
Program Development; Program Evaluation; Program 
Implementation; Staff Development; *Team Training; 

*Workplace Literacy 
* Continuous Improvement 



ABSTRACT 



This guidebook presents a collaborative or team-based model 
of workplace basic education. It is written particularly for 

practitioners- -the adult educators and company and union representatives who 
do the work of planning, carrying out, and evaluating basic skills-related 
activities in the workplace. Chapter 1 provides background, assumptions that 
guided design of the 3 -year workplace education project in seven workplaces 
in upstate New York, and components of the Collaborative Learning for 
Continuous Improvement (CLCI) project model: initial discussion; educational 
planning team; workplace needs assessment; individual needs assessments; 
basic skills-related activities; staff development; program evaluation 
planning; activities scheduling and facilities preparation; learner 
recruitment; implementation of instructional, assessment, and program 
evaluation activities; formative evaluation; and end-of- cycle evaluation. The 
remainder of the guidebook describes how the sites actually interpreted the 
model. Chapters 2-7 describe how the sites implemented the various program 
components and the lessons learned from that experience. These chapters cover 
the following: (1) the educational planning team; (2) workplace needs 

assessment; (3) curriculum: providing multiple learning opportunities; (4) 
portfolio assessment; (5) program evaluation; and (6) the central planning 
team. Chapter 8 provides bibliographic information and descriptions of 34 
reference materials related to topics covered. (YLB) 



******************************************************************************** 



Reproductions supplied by EDRS are the best that can be made 
from the original document. 



******************************************************************************** 




I £? LL0 9-> 



Collaborative Learning 






ace 



ram 



in a 



A Guidebook 
from a Project 



by Paul Jurmo 



August 1998 



U.S. DEPARTMENT OF EDUCATION 
Office of Educational Research and Improvement 
EDUCATIONAL RESOURCES INFORMATION ; 

Y CENTER (ERIC) j 

£0 This document has been reproduced as , 

^ received from the person or organization j 
originating it. . 

□ Minor changes have been made to r* 

improve reproduction quality. |. 



Points ot view or opinions stated in this f . 
document do not necessarily represent 
official OERI position or policy. J ■ 



Produced by the New York State Education Department under a grant (Project Number 
V198A40318) from the ,U.S. Department of Education. Available from Learning Partnerships, 14 
Griffin Street, East. Brunswick/ New Jersey 08816-4806, 732-254-2237.' 



O 

ERIC 



Collaborative Learning 
for. Continuous. Improvement : 

Team Learning and Problem Solving in a 
Workplace Education Program 



A Guidebook 
from a Project 

of the National Workplace Literacy Program 



by Paul Jurmo 



August 1998 



Produced by the New York State Education Department under a grant (Project Number 
V198A40318) from the U.S. Department of Education. Available from Learning Partnerships, 14 
Griffin Street, East Brunswick, New Jersey 08816-4806, 732-254-2237. 



O 

ERIC 



3 



Contents 



Introduction and acknowledgments i 

Chapter 1 : Where we started: Background, assumptions which guided us, 

and components of the project model 1 

Chapter 2 : The educational planning team (EPT) 5 

Chapter 3 : Workplace needs assessment 15 

Chapter 4 : Curriculum: Providing multiple learning opportunities 23 

Chapter 5 : Portfolio assessment 45 

Chapter 6 : Program evaluation 53 

Chapter 7 : The central planning team 57 

Chapter 8 : References 63 




4 



Introduction and Acknowledgments 



This guidebook presents a “ collaborative " or team-based model of workplace basic 
education developed from 1994 through 1997 in New York State under a grant from the U.S. 
Department of Education. 

The guide is written particularly for * practitioners " -- the adult educators and company 
and union representatives who do the work of planning, carrying out, and evaluating basic skills- 
related activities in the workplace. The project and the guidebook make no pretentions about 
having "the perfect model" which should be replicated in all other workplace education settings. 
Rather, our statewide team attempted -- through a three-year research and development 
project - to field-test some ideas and then share our experience with others. 

We have attempted to make this document accessible (with a user-friendly tone and 
format) and useful (with plenty of examples and resource persons and references for follow-up). 
The project team (listed in Chapter 7) hopes that you feel free to adapt our ideas. We also 
welcome questions and feedback. 

The author thanks the funder (the U.S. Department of Education); the coordinating 
agency (the New York State Education Department); the adult educators, employers, and union 
representatives (listed in Chapter 7) who used their creativity to make things happen at the sites; 
and the learners who participated in this project. 

We present our experience with the hope that this important form of adult learning, 
organizational change, and economic development will get the support it needs. 
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QJAEER 1 

Whore. We Started : 



Background, 

Assumptions which Guided Us, 

and 

Components of the Project Model 



Background 

In November 1994, the New York State Education Department (NYSED) launched 
"Collaborative Learning for Continuous Improvement" (or "CLCI"), a three-year workplace 
education project in seven workplaces in upstate New York. Funded by the National 
Workplace Literacy Program of the U.S. Department of Education, this project was designed to 
test a team approach to workplace education which linked adult education methods to 
companies attempting to shift toward a high performance organizational model. 

The seven sites which participated in the project were: 

0 Albany International in East Greenbush (manufacturing of paper machine fabric) 

0 Albany International in Menands (manufacturing of paper machine fabric) 

0 Albany International in Homer (manufacturing of monofilaments for paper machine 

fabric) 

0 Eastman Kodak in Rochester (photographic products manufacturing) 

0 Delphi (General Motors) in Rochester (automotive manufacturing) 

0 EG&G Wright Components in Phelps (aerospace parts manufacturing) 

0 Elmira Stamping in Elmira (metal stamping products) 

Assumptions which guided us 

This seven-site project was coordinated by a Central Planning Team (CPT) composed of 
representatives from the sites, the project coordinator (the New York State Education 
Department's workplace education specialist), two evaluation consultants, and a curriculum 
advisor. The CPT provided a forum in which key players in the project could define objectives 
and concepts to be tested in the project. 
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In designing the project model, the CPT agreed to try out a team-based problem-solving 
curriculum linked to a high-performance organizational development model. CPT members 
hoped to implement a "team problem solving" approach at three levels: in site-level educational 
planning teams, in the classroom at each site, and across sites via the CPT. 

The CPT wanted to test some ideas which we had previously developed ourselves and 
which we had seen others developing in the fields of adult basic education, workplace education 
and training, and organizational development. Here are some assumptions we based this new 
model on: 

0 Participating employers were interested in problem-solving, teamwork, and higher-level 
SCANS competencies and not just low-level reading, writing, and math skills. ("SCANS" is a 
list of work-related basic skills developed by the Department of Labor). 

0 Participating companies were trying to move toward a "high-performance" model, 
emphasizing teamwork and problem-solving. 

0 Good workplace education practice requires . . . 

. . . collaborative learning (learning and decision-making in teams, reflective of how 
decisions are made and knowledge is shared in the workplace ); 

. . . o broader mix ofbgsjc skills (rather than focusing solely on a few isolated reading, 
writing, or math tasks); 

. . . integration of skills with workplace, improvements and employee development (to 
make learning relevant to both the employer and employees); and 

. . . customized, measures (Each site was to develop its own ways of assessing learner 
needs and interests and documenting progress. Sites were not required to use any kind of 
standardized reading test, as it was felt that those kinds of tests did not measure the more 
complex kinds of contextualized skills to be focused on in the project. Sites were 
encouraged to try to adapt a portfolio assessment model to their particular situations, and 
to include group and individual learning projects as evidence of individual achievement.) 

0 Employers, learners, and other stakeholders need ta be involved in the process of setting 
goals, planning learning activities, and monitoring them. 

0 Companies need to suggo[t what learners learn, or it will likely be forgotten. (Companies 
need to internalize and continue to support learning.) 

0 The education providers had o commitment to the CLCI model and the skills and creativity 
required to carry it out. 

The CPT wanted to go beyond conceiving of workplace basic skills as "reading work 
orders" and move into the more reflective and complex kinds of skills which employers were 
asking for. These were also the kinds of skills and knowledge described in the SCANS report 
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and similar documents. 

In putting together the project, NYSED attempted to identify companies which had an 
active interest in the team problem-solving curriculum which NYSED had in mind. Participating 
companies were also assumed to be moving toward a "high-performance" organizational model 
which emphasized employee education, teamwork, quality, and related concepts. It was hoped 
that a participatory, team-based, problem-solving curriculum would naturally fit with such 
workplace improvement efforts. 

Components of the CLQ model 

CLCI drew on models of team-based workplace education developed in Massachusetts, 
Canada, and elsewhere. The model emphasized stakeholder involvement in identifying learning 
needs, setting goals, designing and carrying out learning activities, and designing and implementing 
activities for individual assessment and program evaluation. By involving stakeholders in these 
ways, it was hoped that those stakeholders would better understand and support the education 
program and make sure it stayed relevant to their needs. 

Stakeholders at each site would be organized in educational planning teams (EPTs) which 
would go through a process of planning, implementing, monitoring, fine-tuning, and reporting on 
program activities. In many cases, sites would repeat these activities a number of times over 
three years, as they completed one set of learning activities and then went on to plan another 
round. These activities would also often be going on at the same time and thus the following 
program components should not be seen as linear but more as a recursive cycle or spiral. 

Here are the activities which educators and representatives of other stakeholder groups 
would be responsible for at each site: 



7. Conduct jnitiaL discussions, to identify who at the site is interested in exploring setting up an 
employee education initiative. 

2. Create an educational planning team EPT). 

3. Conduct a workplace needs assessment (WNA) to clarify: 

(a) the company's and workers' goals for improvement; 

(bl factors which block or support progress toward those goals; 

(c) whether and how basic skills is a factor; 

(d) whether and how basic skills-related activities might help the company and workers 
meet their improvement goals; and 

(e) how basic skills-related activities might be integrated with other improvement initiatives 
which the company and workers are involved in. 
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4. Conduct individual needs gssessmgnts to clarify where individuals "fit" in the above 
assessment of organizational needs and possible basic-skills-related activities. 

5. Develop basic-skills-related activities based on that needs assessment. 

6. Carry out staff development activities, to hire, prepare, and support staff to carry out the 
basic-skills-related activities. 

7. Plan a program eygjuglion strategy. 

8. Schedule activities gad prepare fgcjHtjes - 

10. Conduct legjrier recruitment 

11. Implement instructional, assessment, and program evaluation activities . 

12. Conduct ongoing monitoring of progress and fine-tuning of activities (fgrmatjye evaluation ) 

13. Carry out an end-of-cvcle evaluation to identify program impacts and best practices and 
make decisions about "next steps. * 



What actually happened 

Shown above are assumptions and components which we field-tested over three years. How 
(and how well) these ideas worked varied from site to site. 

The remainder of this guidebook describes how the sites actually interpreted this model. Chapters 
2 through 7 describe how the sites implemented the various program components and the lessons 
learned from that experience. Readers are also invited to read the evaluation consultants' 
reports (See Chapter 8.) for discussions of what actually happened when the above assumptions 
were tested in reality. 
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CHAPTER 2 



The Educational Planning Team 

IEPTI 



Chapter 1 refers to an upwardly-mobile “spiral" of program components or activities 
which each project site was to presumably carry out during the three-year project. On that 
spiral, one of the first things a site was to do was to pull together an educational planning team 



(EPT). 



This chapter presents (a) the rationale behind the educational planning team concept, 

(b) the guidelines we prepared for setting up an EPT, and (c) how the sites interpreted those 
guidelines. 

Why use an educational planning team? 

The Central Planning Team felt that a site-level multi-stakeholder planning team was vital 
for these reasons: 

0 To succeed, workplace education programs need to have the active involvement of all 
kev stakeholders : worker-learners, managers, supervisors, union representatives, and 
education providers. Their involvement helps the program to be a true partnership. 
Through ongoing communication with other stakeholders, education staff can more 
effectively identify and respond to those stakeholders' interests; in turn, those 
stakeholders can more easily understand and support the program. 

0 To facilitate this involvement, programs need user-friendly wavs of involving busy 
stakeholders . Just as they need to prepare for their work with learners in the classroom, 
education staff also need to prepare carefully for how they communicate and work with 
other stakeholders. 

0 An education planning team composed of representatives from the various stakeholder 
groups can provide a forum for communication among stakeholders. A well-organized 
team can go through a process of needs analysis, goal-setting, planning of education- 
related activities, implementation of those activities, and ongoing monitoring and fine- 
tuning. 

0 This process borrows techniques from "continuous improvement" approaches to 
organizational development . By going through a collaborative decision-making process, an 
educational planning team can: 

-- Ensure a high-quality, meaningful education initiative; 

- Develop skills and relationships which they can apply to other workplace needs. 
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Guidelines for setting up and running an educational planning team 

The Central Planning Team provided training and guidelines to each site, to help them get 
their EPTs up and running. This section explains what the model recommended in terms of . . . 

0 Who should be on the team? 

0 What criteria do you use to select members? 

0 How do you get them interested and negotiate their roles? 

0 What do you do with them in your first meeting? 

0 How do you make sure that all members can participate 
in a meaningful, productive way? 

0 Who will serve as leader? 

Who should be on the team? 

An educational planning team should have a mix of the stakeholder groups having an 
interest in the workplace education effort. (We define "stakeholder" as an individual or 
institution which is investing something [time, money, or other resources] in the education 
effort and will expect something in return. Note that, in current organizational development 
parlance, a "stakeholder" might also be termed a "customer" of the education effort.) 

Stakeholders might include workers (both program participants and other workers), union 
representatives, higher-level managers (production and human resource/training managers), 
supervisors, and education providers. If an outside funder like a state government is 
involved, a representative of that funding agency might also be considered a stakeholder. 

What criteria do you use to select members? 

For a team to succeed, members need to meet the following criteria: 

Interest (motivation) Members need to see a value in the education effort and want it to 
succeed. They need to see the EPT as a means of ensuring a successful education effort. 

Time Members need to have a reasonable amount of time to give to the real work of the 
team. Without it, work doesn't get done -- or is done by only a few members -- and the 
team isn't really a team. 

Expertise While members don't have to be trained as "team leaders" per se, they do 
need certain kinds of knowledge and skills. For example, they need to know something 
about the workplace itself and the workforce. They need to be able to perform such 
functions as brainstorming, recording ideas, and organizing information. While some of 
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these skills can be developed through special training, the work of the team will be sped 
up if members already have at least a basic level of these skills when they join the team. 

A mix of workplace roles and backgrounds As noted under "Who should be on the 
team?" above, members should represent a range of workplace roles (jobs). They should 
also be a representative mix of ethnic and linguistic groups, genders, ages, and abilities. 



How do you get them interested and negotiate their roles? 

If you are an "outsider" (an education provider who will be coordinating the team's 
efforts), you will likely have to rely on a few "inside" key contacts to help you identify 
potential members for the team. Go over the above criteria with your key contacts and 
identify some likely candidates. 

Meet with those candidates and explain that you are looking for people who have the 
time and interest to organize and oversee a workplace education effort. Explain the steps 
that the team will go through. (See Chapter Vs review of the program components an EPT 
would oversee.) Make it clear that members need to put some time into the team, but that 
the exact responsibilities of members are flexible. It will be up to the team to decide who does 
what. 



What do you do with them in your first meeting? 

When you get members together in your first meeting, you might briefly summarize for 
them what you had told them individually. Better yet, ask members to tell you what they 
anticipate the team will be about, based on the discussions they already had with you. Ask 
them the following questions: 

0 At this initial stage, what do you think the purpose of the education planning team is? 

0 What do you understand the duties of members will be? 

0 What questions do you have about the team at this point? 

Then present members with your own thoughts on these questions. 

You might then go into some more depth about the "team concept" which you hope will 
underlie your efforts. Rather than lecture them about the history and meaning of teams, first 
get them to think about their own experience on teams, in groups, etc. This will give 
everyone a clearer picture of "where people are coming from" and elicit questions they might 
have about working in a team. To do so, ask them these kinds of questions: 

/. What does "team" mean to you -- what have your experiences been working with 
teams? 

2. What are some advantages and disadvantages of working in a team or group? 

3. What is required to make a team work well? 

4. How is the idea of "teamwork'' now being used in this workplace? 

5. How do you think the notion of "team" might be applied in a workplace education 



project? 



Note the participants' responses. This will serve as a record of the early thinking of team 
members. 

How do you make sure that all members can participate in a meaningful, productive way? 

The fact that someone joins a team doesn't necessarily mean that he or she will 
automatically feel comfortable and be able to be an active participant. Ideally, all members 
will be able to have a high degree of responsibility, control, and reward vis-a-vis the group's 
activities. 

Some members, however, might lack some of the skills or the self-confidence needed to 
participate in a team. In some cases, there might be a history in the organization of some 
groups of employees not being empowered to do the kinds of things you now want them to 
do: speak up, take risks, make decisions. 

You need to -- in a diplomatic way -- make it clear that all members should be encouraged 
to participate actively. Make it clear that they can talk with you privately if they feel 
inhibited from participating actively -- for any reason. 

You also have to be sensitive to group dynamics and structure discussions so that all 
members get an opportunity to speak. In multilingual workplaces, you might need an 
interpreter or have members who aren't fluent in the dominant language form smaller groups 
in which they can speak freely in their own tongues. You might have to limit the amount of 
reading and writing tasks for members who don't feel comfortable with print. Members might 
take turns preparing presentations or serving as recorder, so that all get a chance to play a 
variety of roles. 

In addition to structuring day-to-day team activities in ways to nurture full member 
participation, you might also organize special events to instill a team identity and open 
communications. You might, for example, arrange an informal lunch, picnic, or evening at the 
ball game. Perhaps the company has an annual volleyball tournament and your team could 
participate. You might, if resources permit, go to a special "team training" workshop or 
retreat. 

Just be sensitive to the fact that you will likely have to continually nurture a team identity 
and infrastructure to enable members to participate fully. Don't expect members to 
automatically begin operating as a cohesive unit just because you've gotten them to sit down 
together and call themselves a "team." 

Who will serve as leader? 

The role of leader -- or "facilitator" or "coordinator" -- of an educational planning team is a 
vital one. This is the person who must do most of the "grunt" work of organizing meetings, 
keeping track of records, writing reports. 

In fact, the team might divide this work among several different people. The more the 
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work of the team is shared, the more likely it is that members will take ownership for the 
group. This will also reduce the likelihood that one person will burn out from overwork. 

The leader(s) of the team must have all of the characteristics described above under 
"What criteria do you use to select members?" Leaders must also have qualities like patience, 
perseverance, an understanding of democratic principles and procedures, willingness to listen 
and encourage good ideas, ability to communicate clearly, and ability to balance the many 
interests represented in the group. Bookstores are full of "how to be a good leader" guides. 
You might read a few of them. 

Keep in mind that, for an educational planning team to work, good leaders are vital. The 
leader is the person who drives the effort and keeps it on track. This is not easy and requires 
a sustained, committed effort. Whoever takes on that role should think carefully about the 
responsibilities entailed and prepare carefully to fulfill them. 



How the CLCI sites interpreted their educational planning teams 

Each of the CLCI sites interpreted the idea of a multi-stakeholder educational planning 
team a bit differently. The teams varied in terms of their make-up, how involved members 
got, and what they focused most of their attention on. These factors were, in turn, 
determined by things which were in some cases beyond the education coordinator's control 
(e.g., a plant might be undergoing downsizing and, therefore, team members' attention was 
on matters other than education) and, in some cases, within the educator's control (e.g., the 
"marketing" skills of the education coordinator). 



For example, when the education team at E.G.&G. Wriaht Products (an aerospace parts 
manufacturer) was first set up in 1995, the team was composed of the coordinator, the 
company's human resource development manager, and representatives of various 
departments. Over time, the make-up shifted to include more representatives of higher- 
level management, including the managers of the business unit (i.e., the plant manager), the 
quality unit, and the purchasing unit. Education coordinator Dianne Spang feels this was due 
to management's recognition that the EPT was adding value to plant operations, an efficient 
way of identifying and responding to priority needs in the plant. A "white board" tracking 
system and "PIF" sheets were concrete results of the initial communications course run by the 
EPT and were seen by management as concrete, useful products of the EPT and the program 
overall. Because they saw the EPT and the education program as useful, EPT members met 
monthly and looked at ways to use the federal grant to improve the company. 



At the Albany International plant in Homer, site coordinator Paula Hayes worked with 
management representative Linda Holland to organize an EPT in early 1995. The EPT carried 
out the workplace needs assessment (WNA) described in Chapter 3. Based on the WNA, 
the team spent the summer of 1995 developing a curriculum (described in Chapters 3 and 4). 

The EPT became less active by fall 1996, however, due to three factors: (1) the EPT's 
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primary focus -- organizing, carrying out, and acting on the WNA -* was completed by that 
point; (2) the EPT's key management representative, Linda Holland, was given a new position 
which required her to focus more on production than human resource issues; and (3) workers 
were less available to interact with the site coordinator due to increased production and due 
to an accident which occurred in spring of that year. By 1997, the EPT was reduced to four 
members, including the site coordinator. 



When CLCI began in 1995, Eastman Kodak's Rochester facilities had for several years 
been making the transition to a team-based, "high-performance" organizational model. Also, 
the new company CEO was placing increased emphasis on employee training. The CLCI 
program model was seen as a vehicle for helping the company meet those goals. 

The education provider, Rochester City Schools (RCS), had for a number of years 
provided various kinds of basic education services to Kodak employees. RCS staff thus were 
able to build on the relationships and understanding of the company culture and its 
educational needs which it had already established at the company. RCS and Kodak 
representatives agreed to form a modified version of an educational planning team composed 
primarily of RCS teachers and the training advisors from several company departments. This 
team helped with the initial planning of the program and then met every two weeks to 
monitor progress. 

The make-up of the team changed over time but at various times included teachers, a 
RCS administrator, training coordinators for three company units, and the human resources 
director for the department within which the program was being carried out. These changes 
came as the company shifted personnel into new jobs, but the EPT's role in monitoring and 
coordinating the program persisted. The EPT scheduled classes; strategized ideas for new 
classes that could fit the business' needs; tracked results for the company and for the external 
evaluator; and addressed issues that arose within the classes or out on the floor. 

In the initial EPT meetings, RCS staff explained to Kodak training staff the team problem- 
solving model being developed under the grant. The company representatives considered 
whether and how the grant's problem-posing model fit with the company's training needs and 
concluded it did. Company representatives had already embarked on a team-oriented 
continuous improvement initiative and saw how the problem-solving classes could serve that 
effort. The EPT thus decided to go ahead and try the instructional model in problem-solving 
classes for three "natural work teams" already existing in the facility. In their initial meetings 
with RCS staff, the three classes (each from a separate work area) identified a number of 
workplace problems to focus on. 

The Kodak site did not carry out a formal workplace needs assessment in the way 
proposed in the grant. It was later found that such a broader assessment might have helped 
identify other educational needs and obstacles within the organization which could have been 
factored into program planning. 

RCS staff concluded that the EPT "was an essential tool to maintain customer focus and 
workplace involvement." RCS "absolutely" intends to continue to use EPTs because . . . 



... in circumstances where the business is focused on implementing significant changes 
in human resource initiatives, and they understand the value of training as a vehicle 
for successful initiation of the changes, an EPT is necessary. An EPT can predict 
potential problems which may occur in the running of these classes, plus it can keep 
the pulse of the environment so as to make changes when they are necessary. 



The Delphi plant manufactures fuel injection systems for General Motors trucks. When 
the CLCI project got underway in 1995, the plant was in the midst of creating a new product 
line, with new work processes and new work teams. That line was to have volunteer 
employees and require better communication and attention to quality. (GM told the plant it 
would buy its fuel systems elsewhere if the plant didn't meet quality standards.) The basic skills 
program was seen as a tool for helping the plant make that shift. 

Rochester City Schools (RCS) already had a history of working in this plant. Under this 
new federal grant, RCS staff worked with an existing team which had been set up to 
oversee the plant's transition to the new product line. That team had already gone through a 
strategic planning process which resembled the CLCI model's workplace needs assessment 
process. Rather than undertake an entirely new WNA, the existing planning team in early 
1995 did a modified workplace needs assessment, reviewing the strategic plan which the 
union-management team had already developed, to clarify where the CLCI problem-solving 
curriculum might fit in. The EPT decided to focus the first round of instruction on helping 
workers to perform statistical process control (SPC) calculations. 

After those initial SPC classes were carried out (See Chapter 4.), RCS brought in a new 
instructor in fall of 1995 to replace the original site coordinator, who had gone on maternity 
leave. He was to work with another product line department, one which had worked 
together for some time. This new coordinator, Tom Wager, realized that he needed to 
quickly get up to speed, to get to know the EPT and issues within the department, and to 
clarify what to focus the next phase of services on. Tom and the EPT decided that he should 
do a modified workplace needs assessment. (See Chapter 3.) 



The EPT at the Albany International site in East Greenbush started off with a half dozen 
members and then dwindled to two members who carried out most of the work. Other 
members became less active due to changes in their job descriptions and an increased 
production load. This reduction in active membership was considered appropriate in this site, 
however, because while the larger team was needed initially to identify needs and get 
stakeholder buy-in, it was felt that only a few people were needed to perform the nitty- 
gritty tasks (of planning lessons, for example) which followed. Further, the coordinator 
appreciated having the freedom of being able to work on her own as she, by trial and error, 
tailored a curriculum relevant to the learners. 

The down side of this was that there was not a ready-made audience for the coordinators 
to communicate with as the problem-solving classes began to produce results. For example, 
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the coordinators would have liked to have been able to give updates to an active, informed 
EPT about what the workers were saying about communication problems within the company. 



Lessons learned about educational planning teams 

Of the seven project sites, six were successful in implementing the EPT concept. Based on 
their experience, those six sites generally state their support of the concept, saying that it 
was vital in helping the providers understand the interests and issues represented in the site, 
and in involving stakeholders in supporting and participating in the program. (The seventh 
site was essentially not ready to participate in the project and never really took the step of 
setting up an EPT.) 

As described above, implementing the EPT concept meant different things in the six sites. 
For example, the make-up of the teams and the roles members performed varied considerably 
from site to site and even within individual sites over time. Most EPTs had members 
representing several company departments and several job levels (white collar, blue collar). 
Which departments, of course, depended on the company, its size, its needs, and in some 
cases who was in charge of a particular department. (People have to have time, awareness, 
and interest to be able to participate actively in an employee education initiative.) 

At the end of the three years, site representatives gave the following feedback about 
their experience with the EPT concept: 

The value oj access . One coordinator concluded that a key ingredient for a successful 
workplace education program is " access to workers . " She says: 

I discovered that I had a tremendous advantage at this company in being able to 
wander freely through the building, sit in the break rooms, schedule brief meetings 
with "students" during work hours, etc. 

The EPT gave her access not only to representatives of the various stakeholder groups in 
the company but to the Constituents’ of those representatives out on the shop floor. 

Personalities and, power relationships . One coordinator felt that any group like an EPT 
will likely have personalities and power relationships which an outside educator can't 
know about in advance and which could make or break support for the educational 
program. The EPT, nonetheless, provides an opportunity for the educator to get to know 
these people, the dynamics of decision-making in the organization, and issues which can 
be woven into planning of the education initiative. 

EolenliaL conflicts wjihjn ths. group . The same coordinator recommends that any educator 
entering a workplace should realize that consensus within an EPT is not a "given." The 
coordinator should be prepared to deal with conflicts, grievances, or lack of agreement 
within a group composed of different organizational stakeholders. It is important that a 
group, from the start, be up-front about potential obstacles to working together, so that 
the educator and the group don't set themselves up for unnecessary frustration. 
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EPTs gs o forum /or degjjng with 'sticky issues . " One coordinator found that, by opening 
the classroom up to discussion of workplace problems, it was inevitable that groups would 
identify factors within the organization which undermine full worker participation in 
organizational change. She wondered whether and how other EPTs have done anything 
about those organizational obstacles if they have uncovered them. 

Being consistent with how stgkehojderi vjew collaboration . One coordinator notes that it 
was easy for her to use the EPT concept because her agency had historically always 
emphasized collaboration with the companies and unions it worked with. Similarly, the 
EPT model works best in an organizational context which encourages employee 
participation in problem-solving. 

The coordinator feels that, in the future, she will be more selective about what 
kinds of companies she will try to use an EPT model with. For an EPT to work, a company 
has to be willing to invest the time to make it work. If a collaborative approach is to be a 
feature of future government-funded programs, she would make it a requirement and 
negotiate it in her communications with sites. At the same time, an education provider 
needs to be flexible and not force anything down a company's throat. 

"Selling’ the EEL concept . One coordinator felt that the EPT was very effective in terms 
of generating interest and buy-in of all the right players in the plant. However, for an 
EPT to work, all those players need to be ready and willing to participate fully in the EPT. 

She feels that a coordinator needs to be able to "sell" the EPT concept to potential 
members, as this is a new concept for most of them and they are likely to want to know 
"what's in it for me?" 

The coordinator recommends that "marketing your program" be included in 
training given to workplace educators, to ensure that they know why and how to "sell" 
the EPT and other concepts required for a good program. 

Involvement oL workplace leaders is particularly important as a way of helping them to 
understand and support the program. One coordinator cited two work team leaders who 
were involved in the program's planning and early activities from the start. Those two 
were positively impressed by the program's impact on opening up communication channels 
and subsequently helped to transfer this group-learning process to how their teams 
operate out on the floor. 

In one site, a new manager showed real interest in the program by writing letters 
of encouragement to participants. In another case, company managers bought into the 
problem-solving curriculum model and paid attention to the reports which the learners 
generated in their classes. 

Funds and flexibility . One coordinator stated that it is also vital that a coordinator have 
enough funds and flexibility to cover the upfront preparations which this model requires. 
Decision makers need to understand that a good program typically can't be started 
immediately, from scratch, without adequate preparations. 
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CHAPTER 3 



Workplace Needs Assessment 
(WNA) 



As shown in Chapter 1 in the list of components of the CLCI model, a workplace needs 
assessment (WNA) is the first major activity which an educational planning team should 
undertake. 

This Chapter (a) presents the "what, why, who, and how" of the workplace needs 
assessment process, (b) summarizes how the WNA was interpreted in our project sites, and (c) 
discusses lessons learned about workplace needs assessments. 

For more*in*depth guidance about the WNA, the reader is referred to Collaborative 
Needs Assessment: A Handbook for Workplace Development Planners by Sue Folinsbee and 
Paul Jurmo. For more information about how the project sites used WNAs, see the project 
evaluation documents. (See the References cited in Chapter 8.) 



The "what, why, who, and how" of workplace needs assessment 

To help the site educational planning teams prepare to conduct workplace needs 
assessments, the Central Planning Team presented the EPTs with the following information in 
workshops and written documents in the early months of the project: 

What is a WNA? 

A workplace needs assessment (WNA) is a systematic way of identifying workplace basic 
skills needs. 

It identifies those organizational needs that might be met through educational activities 
and those that need to be addressed through other changes in organizational policies and 
practices. 

In particular, it documents where basic skills-related activities are needed. 

Whv do a WNA? 

By specifying a range of educational, training, and other organizational development 
activities which an organization might undertake, a WNA ensures that realistic expectations 
are set for educational activities. 



Rather than assume that a single basic skills or other activity can by itself meet all of an 
organization's needs, stakeholders will have a clearer picture of what role education can 
reasonably play. 

A WNA can also identify organizational needs which can become topics for basic skills 
and other education and training activities. 

A WNA can also build awareness, ownership, and support for an educational initiative 
among all levels of the workforce. 

Who would conduct it? 

A WNA would be the first major activity conducted by an organization's educational 
planning team. Such a team would oversee not only the WNA but all other components of 
the education program and related organizational development activities. This team would be 
composed of representatives of key stakeholder groups: workers, managers and supervisors, 
unions, and education providers. (See Chapter 2 for more about the responsibilities and 
make-up of the educational planning team.) 

How do a WNA? (What steps are involved?) 

To plan and conduct a WNA, an educational planning team should: 

7. Develop o aoa/ statement for the. WNA . Clarify what the team hopes to achieve by 
conducting a WNA. (See "Why, do a WNA?" above for possible purposes .) 

2. Clarify what information the team wants to collect . Typically a WNA tries to answer 
these kinds of questions: 

1. What are the company's goals in terms of improving workplace operations? 
What do workers hope to achieve in terms of their career development? 

2. What changes have occurred within the company in recent years? 

a. How have products and work processes changed? 

b. How has equipment changed? 

c. What if any new safety or industry [egulgtjons is the organization 
responding to? 

d. How has the makeup of the, workforce changed? 

3. What factors (inside the company and outside it) currently block or support 
progress toward the company's and workers' goals? 

4. To what degree and in what ways are employees ' basic skills a positive or 
negative factor? 

a. Do employees need to upgrade their communication, thinking, and 
problem-solving skills to respond to those changes? If so, give examples of 
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areas in which employees appear to be having problems. 

b. Do employees recognize these as problems, or is this primarily a 

management-level concern? 

5. What is. the, organization currently doing to help employees respond to the 
challenges they face? What are employees themselves doing to respond to these 

changes? 

6. In particular, might bask, skills-related activities help the company and workers 
meet their improvement goals? 

7. How gijght basjc skills-related gctiyjtjes be integrated with other improvement 
initiatives which the company and workers are involved in? 

3. Decide where you'll aet vour jnfgnrigtjgn gnd design information -gathering activities . 
WNAs typically use interviews, focus groups, and possibly questionnaires to collect the 
information they need from a representative sampling of the various stakeholders 
represented in the organization (and on the education planning team). Information might 
also be gleaned from documents (e.g., annual reports, strategic plans, and education and 
training records ) and through observation of workplace and education/training activities. 
Help the team decide how it will collect the desired information from those sources. Field- 
test your activities and refine them. 

4. Collect the, information you need using the, activities you designed - 

5. Organize gnd summarize the, information, collected . 

6. Analyze thgt information gnd prepare o report with recommendations . 

7. Present your report gnd octree on gcjjons to take . 

8. Prepare gn gcjjgn plan and follow up to ensure that actions are taken . 



How does a WNA differ from other types of assessments done in workplace education 
programs? 

Until recently, assessment in workplace basic skills programs has tended to follow a model 
developed in school settings. In this academic model, assessment focuses solely on the 
individual and tries to determine the individuals ability to perform certain tasks. 

More recently, workplace educators have used "literacy task analysis" (a "literacy audit") 
to analyze the basic skills requirements of particular job tasks and then assess whether 
workers could meet those skill requirements. 

A WNA broadens the focus of assessment to include the organization within which the 
individual (in this case the worker- participant) operates. A WNA assumes that a healthy 
organization requires continuous improvement (change) by not only individual employees but 
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also by other stakeholders in the larger organization. 

A WNA is a procedure for assessing the organization's ability (readiness) to effectively 
use the knowledge and skills of its workforce. When combined with assessment of individual 
workers' abilities, a WNA can give a comprehensive picture of what the organization and 
the individual workers need to do to meet their goals for continuous improvement. 



How the CLCI sites interpreted workplace needs assessment 

Four of the project sites conducted fairly extensive WNAs. Two others felt that they 
already had a good sense of what the companies and workers needed based on previous 
strategic planning carried out in the companies and on the education providers' prior 
experience working in the sites. 

Here is a summary of how the sites used the WNA methodology: 



D A t Albany International's East Greenbush plant, the coordinator worked with her EPT 
to conduct an extensive WNA. The coordinator conducted many site visits, interviews, 
focus groups, observations, and reviews of documents. She relied in particular on the 
input and insights of a human resources manager who had a special interest in what it 
takes to shift an organization into a "high performance" mode. The WNA report 
identified "communications" as a focal point for the first round of learning activities. The 
process showed that the plant was, like many others, struggling with the question of how 
to continue its shift toward a "quality team" environment. Stakeholders saw the CLCI 
program as a tool for helping the organization to resolve that question. 



D The site coordinator at the Albany International plant in Homer made many visits - 
often during early morning hours *■ to the plant and talked with individual workers when 
they could spare a few minutes on the shop floor. The resulting WNA report indicated 
that the company initially wanted to focus on helping extrusion machine operators 
understand where paperwork fits into the production process and to deal with particular 
paperwork tasks required in their jobs. The paperwork allowed workers to document 
noncomforming products and problems which lead to defects. The paperwork was a 
combination of narrative descriptions, checklists, and computations filled out by workers. 
The company also wanted the education program to help improve teamwork, 
communications, and problem solving among workers. The EPT anticipated that additional 
needs would emerge once instruction around the above objectives got underway. 

D At the Albany International plant in Menands. the site coordinator worked with the 
plant's EPT to conduct a WNA in early 1995. It showed that the plant's workforce 
consisted of experienced workers with strong technical skills and a high quality ethic but 
many lacked sufficient "softer" skills required in a team environment. The WNA thus 
indicated a need for educational activities which helped employees better work in teams, 
communicate, and solve problems. 

18 



o 

ERIC 



22 



When employees were asked what workplace problem(s) a communications and 
team problem-solving program might focus on, they responded that many workers were 
having trouble understanding the company's financial benefits package. The focus for the 
first round of instruction thus used the company's insurance and retirement benefits 
package both to teach reading, writing, math, and problem-identification and -solving 
techniques, and to provide opportunities to practice proactive teaming skills such as 
listening and negotiating. 

Put another way, the program was to help workers develop the skills they need 
to be problem solvers and process improvers. 



D When the program began at the Delphi plant in Rochester, the EPT did not conduct a 
formal WNA per se. The team felt that they already had a good grasp of what basic 
education-related activities were needed because of previous strategic planning activities 
and because the education provider had a considerable first-hand knowledge of the plant 
from earlier education programs they had provided there. Based on this prior needs 
analysis and experience, the EPT agreed that, in the first round of educational activities, 
the program should focus on math skills needed to carry out statistical process control 
(SPC) functions. 

As that SPC course wound down, another instructor joined the program. He was 
to work with another product line department, one which had worked together for some 
time. As a newcomer working with a department which hadn't previously been involved 
in this federal project, the instructor and the EPT decided that he should do a modified 
WNA to determine what needs to focus on in his curriculum. The WNA indicated that 
the next phase of instruction should help work teams to be more autonomous by 
enhancing the teams' problem-solving and team-building capacities. 

Workers indicated that they didn't simply want "another class” which would teach 
them things they never applied in reality. It was agreed, instead, to structure learning 
activities as problem-solving teams whose members would identify a problem and use a 
problem-solving method to analyze and then solve real problems the workers faced. 
Further discussions identified several technical and communications problems, including one 
with continual rejects which had stymied plant engineers and another of constant re- 
checking which wasted employee time. 



0 At the E.G.&G. Wriaht plant, the coordinator attended corporate training sessions as a 
participant-observer. She thereby familiarized herself with the issues the company and 
workers were dealing with, to ensure that her instruction would meet real needs. Among 
other things, she found that shop-floor personnel weren't getting access to team-related 
training that higher-level employees were. She concluded that the initial round of 
instruction should focus on “listening and reflective response," tying the program in with 
the company's interest in conflict resolution. This process also showed that many in the 
company wanted introductory computer-related training, to enable all workers to handle 
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the plant's computerized information system. The EPT decided to conduct such training in 
the second year of the program. 



Lessons learned about workplace needs assessment 

The WNA's potential . The coordinator at the Albany International site in East Greenbush 
stated that she "loved the WNA and wouldn't change it at all. It's a valuable orientation for 
everyone," especially for an outsider like an education consultant. It "gives background and 
context and it really does integrate the program into the social context of the organization 
and clarifies the readiness of the organization to change." 

In the words of the coordinator at the Albany International plant in Menands, the WNA 
was "great." She feels it was important, as it gave her a chance to get to know individuals 
she would eventually work with. She learned a lot by. going out on the floor and spending 
time with them. 

The same coordinator said that she intends to use the WNA process in future workplace 
education situations. She described the benefits of the WNA concept as follows: 

Designing the WNA within an EPT setting causes the group to work toward a 
greater commonality of focus. The first steps toward resolution of any problem are 
(1) recognition that there is a problem and 12) definition of the problem. Administering 
the WNA in a safe, small-group setting prompts further discussion resulting in 
additional information which can be brought back to the EPT. In some instances, the 
initial problem which the EPT saw as most significant and/or urgent was not the issue 
which the class perceived as most significant and/or urgent. 

The coordinator at E.G.&G. Wright also felt that the WNA was very useful. It helped 
her "get a handle on the culture of the company. As a trainer, you need this to be able to 
respond to their needs." 

The importance of trust . The coordinator at E.G.&G. Wright emphasized that trust is vital if 
informants are going to give accurate, truthful feedback in a WNA. This trust can be 
increased by having the coordinator do an upfront explanation to employees of the purposes 
and guidelines for the WNA before embarking on it. 

Obstacles to conducting a WNA Site coordinators cited several potential obstacles which 
those interested in using a WNA should be aware of: 

Time constraints : One coordinator acknowledges that many educators might not be 
comfortable putting in the amount of time that an extensive WNA requires. This lack 
of comfort might be due to limited time (e.g., getting out on the shop floor at 5:00 
a.m.) or a lack of familiarity with the factory environment or blue-collar populations. 
Nonetheless, the coordinator feels that, though time consuming, individual interviews 
are a very valuable way to collect information and, at the same time, develop a 
relationship with workers. 

She says that, for this approach to needs assessment to work, educators 
need support from both the company and from their own educational institutions. 
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Educators need to be given the time, trust, and flexible work schedule to allow them 
to be on the worksite at odd hours and to do the customized, just-in-time curriculum 
development work and instruction which a good program requires. 

Sexism : One problem encountered in conducting a WNA by a woman site 
coordinator was the fact that "it's a man's world over there" and that a woman like 
herself would tend to find it hard to communicate and be taken seriously. 

Need to be efficient with available resources . One coordinator says that it also helps to 
have had a history of working with companies and to build on the relationships and 
knowledge already exisiting. Otherwise, an outsider must take a lot of time developing trust 
and identifying whom to work with. 

She would also make future WNAs more concise and focused, to avoid redundancy and 
to save time She recommends that an EPT continually update or "revisit" its original WNA 
from time to time, to re-build the EPT and to keep focused on learning needs as they emerge. 

The coordinator at E.G.&G. Wright reported that, due to time constraints, she might do 
more focus groups and fewer one-to-one interviews when conducting WNAs in the future. 
She might also send out print questionnaires or electronic versions, where feasible. 

The importance of the EPT . The coordinator at E.G.&G. Wright said that the EPT was vital 
in making the WNA work, as it identified the questions to ask and then helped carry out the 
survey and confirmed the relevance of the WNA s findings. The EPT provided a forum in 
which employees and managers both agreed on the same learning needs. 

Other challenges . The coordinator at Albany International in Menands noted a number of 
challenges. She says that deciding what questions to ask and how to word those questions 
can be difficult. Administering the WNA is "time and money not spent teaching." And 
"responses may vary widely from unit to unit and shift to shift, reflecting vastly different 
cultures within the same facility, thus making generalized conclusions impossible." EPTs need 
to acknowledge these challenges and be prepared to deal with them. The coordinator says: 

7 believe it's important to have an informed team ~ an EPT for example -- to 
formulate the initial questions, since the direction and the wording of the questions will 
significantly impact the information returned as a result of the WNA. If the focus is 
off base, or if the questions are slanted or loaded, the WNA can become almost 
useless, or even worse, a tool of the administration (or any other single stakeholder 
group) to support (impose) its point of view. " 
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Cymcylum: 

Providing Multiple Learning Opportunities 



After conducting a workplace needs assessment, an educational planning team can 
proceed with planning the various basic-skills-related activities which will help meet the learning 
objectives identified in the WNA. The team should also develop a plan and tools for individual 
assessment and program evaluation. (See Chapters 5 and 6.) 

All of these preparations should go on more or less simultaneously, before instruction and 
related activities get fully underway. This chapter will focus on instructional activities or the 
“curriculum" used in the CLCI sites. As in other chapters, we will here (a) summarize the 
curriculum guidelines which the CPT presented to the sites, (b) describe how the sites interpreted 
those guidelines, and (c) discuss lessons learned from our experience with curriculum. 



The curriculum guidelines developed by the Central Planning Team 

In the initial months of the project, the CPT presented a number of concepts to 
repesentatives from the EPTs during staff training workshops and in a number of handouts. The 
CPT hoped that the EPTs would “run" with those concepts and tailor them to the creation of 
curricula which responded to the learning needs identified in their WNAs. 

Here are the key concepts which sites were asked to work with: 

‘Curriculum": A working definition 

We consider curriculum development as more than merely pulling together a 
collection of workbooks and job-related materials and "teaching" them. It is a more- 
comprehensive process. This is our working definition of "curriculum": 

’Curriculum’ is an ongoing process of identifying learning objectives, designing activities 
to help achieve those objectives, and then implementing and continually improving 
those learning activities. 

Roots of our curriculum model 

This "Collaborative Learning for Continuous Improvement" approach to workplace 
education has been influenced by concepts from the worlds of organizational 
development, adult education, evaluation, and elsewhere. These concepts include: 



From ocggnjzgtjongi development 

The learning organization: 

From Peter Senge: 

0 * Learning has very little do do with taking in information. " Rather, 
learning is "creating and building the capacity to create that which you previously 
couldn't create. " 

0 "Team learning is vital because teams, not individuals, are the 
fundamental learning unit in modern organizations. " 

0 "The discipline of team learning starts with 'dialogue, ' the capacity of 
member of a team to suspend assumptions and enter into a genuine 'thinking 
together'. " 

0 "A learning organization is a place where people are continually 
discovering how they create their reality. And how they can change it. " 

0 "In a learning organization, there is a special spirit of an enterprise made 
up of learners." 

TQM and continuous improvement 

A way of viewing work which challenges all involved in an organization to 
continually question "Whom are we trying to serve? What do they need? What 
do we need to do to ensure that their needs are met?" 

Fligh performance organization 
An organization which: 

0 follows a continuous improvement ethic to produce high~quality services 
and products which meet customer needs. 

0 provides the necessary supports to all employees (i.e., a high quality of 
work life) to enable them to produce high quality services and products. 

From adult education (especially literacy and workplace education) 

Contextualized learning 

A concept from research on how people learn. Holds that learners best develop 
skills and knowledge by applying what they already know to meaningful tasks. 
Through such practice, learners gradually develop their own strategies for 
accomplishing those tasks (i.e., they "master" those tasks). 

Metacognition 

A process of improving performance in any skill area through thoughtful practice. 
The learner reflects on what she already knows about a particular task, practices 
applying one or more skills to that task, reflects on her performance, and refines 
her strategies for future use. 

Whole language 

A concept from psycholinguistics which argues that thinking, reading, writing, and 
oral skills are closely related psycholinguistic processes and should be developed 
simultaneously in integrated, natural, meaningful, applied learning activities. 
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Participatory education 

The notion that adult learners see learning as more meaningful and invest 
themselves more in the learning process when activities are structured to give them 
responsibility, control, and reward for what goes on in an educational program. 

Elements to be factored into the curriculum 

Problem* posing 

A participatory method in which learners identify problems/ issues/ concerns which 
they face, analyze the causes of those problems, and then develop strategies for 
responding to those problems. 

Collaborative learning and decision-making 

A notion developed in workplace education programs which holds that, for 
stakeholders (eg., managers, supervisors, union representatives, learners, educators) 
to support and benefit from an education program, they need to have active roles in 
planning, implementing, monitoring, and improving the program. 

SCANS 

A U.S. Department of Labor listing of five competencies which all U.S. workers should 
have: resources (effective use of resources), interpersonal (working with others), 
information (finding and using it), systems (understanding complex relationships), and 
technology (uses a variety of technologies). Underlying these five competencies are a 
range of “foundation skills" (basic language and math skills, thinking skills, and personal 
qualities), as well. This pushes -- and allows -- workplace education programs to go 
beyond focusing only on a few discrete — and often disconnected -- reading, writing, 
and or math tasks. SCANS better reflects the complex demands actually put on 
workers in the emerging U.S. workplace. 

Portfolio assessment 

An approach to identifying, documenting, and reporting what learners need and 
want to learn, progress toward those learning goals, and what needs to be done to 
further help the learner succeed. Portfolios tend to include real-world ("authentic") 
artifacts (such as sample writings, projects created by the learner, etc.) which 
demonstrate what learners can do rather than standardized tests. Portfolio 
assessment is seen as a vehicle for helping the learner to take more responsibly for 
reflecting on his or her learning. This is in contrast to more conventional assessment 
which keeps control of assessment and learning more in the hands of the teacher. 

Learners in a workplace education program can use portfolios to store evidence of 
how they are using basic skills to deal with real tasks they face in their lives as workers 
(and possibly in their lives outside work, too). Such evidence can be more meaningful 
to workers and employers than standardized test scores. 

What constitutes good practice: Our assumptions 

We borrowed from these concepts and wove them together to produce an 
approach to curriculum based on the following assumptions about what constitutes good 
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practice: 



0 For an employee basic skills initiative to have lasting impact, it cannot ' stand alone. ’ 

It must, rather, be integrated systematically with other education, training, and 
organizational development activities to enable learners to apply what they are 
learning to the continuous improvement of the organization and their own personal 

situations. 

0 Stakeholders in the host organization must have a common vision of the role of a 
basic skills program in the larger process of organizational improvement. They must 
commit themselves to actively participating in the planning and continuous 
improvement of the education program. They must see themselves as * learners ’ who 
will themselves take the time to develop the knowledge and skills needed to make 
such an integrated approach to basic skills education work. 

0 In practical terms, all stakeholders would participate in a variety of well-planned 
activities in which they draw on their existing knowledge and skills and apply them to 
particular continuous improvement tasks. 

+ In the case of the site-level education planning team, members will set goals, 
develop knowledge and skills they need to support the program, help design the 
program, help implement educational and related activities, and continually monitor 
progress and improve the program. 

+ In the basic skills and related educational activities at the ’classroom'' level, 
learners and facilitators/instructors will similarly set learning objectives, develop 
knowledge and basic skills in the context of posing problems and developing action 
plans for resolving those problems, and continually improve the education 

program. 

+ Education staff will similarly work at the site level and across sites with other 
education staff and state-level resource persons to set goals, develop knowledge 
and skills they need to do the job, develop strategies and tools, and refine the 
overall program and activities at the site level. 

D In such a model, all stakeholders at all levels adopt a similar * collaborative learning for 
continuous improvement ’ ethic and procedures. They thereby continually test the 
model and see how it can be adapted and refined at the site level. The basic skills 
program’thus becomes a vehicle for all stakeholders to develop their own knowledge 
and skills related to continuous improvment and, in so doing, develop learning 
organizations within the institutions they work in. 

How the sites interpreted curriculum 

As with other components of the CLCI model, the above curriculum guidelines were 
interpreted differently from site to site. Some sites chose to focus primarily on “problem- 
posing" (i.e., helping learners to identify workplace problems and then map out solutions for 
them, while also developing various “basic skills"). Other sites focused more on particular 
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issues like "improving communications," "math," "introduction to computers," or "understanding 
how to use the company's benefits package." 

Whatever the exact focal point of instruction, "problem-posing"and "problem-solving" were 
in most cases woven into the curricula used at the sites. Within the CPT, there was initially 
some colleaguial tension about how much to require the sites to focus on "teaching the 
problem-posing/problem-solving method" versus encouraging sites to respond to whatever 
learning needs emerged from their workplace needs assessments. In effect, the CPT realized 
that there was both a value in focusing on problem-solving as a SCANS competency while 
also being true to the workplace needs assessment process which they had been asked to 
use. 

The CPT eventually compromised and agreed to encourage the sites to provide instruction 
which responded to whatever basic skills needs were indentified in the WNAs, but to put 
special emphasis on incorporating problem-solving techniques wherever appropriate. 

This section describes how the sites interpreted these curriculum guidelines. 



Problem-solving at Eastman Kodak 

The content and formats used jn ths. curriculum 

The classes at Kodak merged the CLCTs version of a problem-posing model with 
Kodak's own internal problem-solving process which, in turn, drew heavily on the 
problemsolving model developed at Cornell University. The Kodak classes were to 
analyze the causes and possible solutions for various problems and develop action plans to 
present to the EPT and management representatives. 

In one class, learners dealt with the question of whether and how workers can be 
rotated to different jobs. The team concluded that women were often not able to handle 
some of the heavier jobs historically assigned to men, and they recommended a way to 
ensure that workers were only rotated into jobs they could handle rather than to expect 
every worker to be able to handle every job. This was seen as primarily a "social" rather 
than “ technical " problem. 

Another class looked at the related question of how to ensure the quality of the 
film containers it produced, how to track the cost of scrap, and how to reduce the amount 
of scrap. This was seen as a problem which was more " technical in nature. 

Another team dealt with the resistance of some people to work with others in a 
team situation (a "social" problem). 

From this experience, one instructor concluded that the problem-solving model has 
great potential but that it might be best to focus - at least initially - on problems which 
are more "technical" in nature rather than on “ social " (interpersonal, cultural) problems. 

The latter are typically "sensitive" and uncovering them can lead to discord and hurt 
feelings which can then lead learners to not want to participate in the problem-solving 
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classes any further. 

As these problem-solving classes proceeded, it became apparent that some 
learners had particular basic skills problems (e.g., lacked basic reading, writing, math, or 
ESOL skills I which could not be dealt with effectively in the classes. RCS staff also felt 
some workers could benefit from "learning-to-learn "-type activities, including identifying 
one's own learning strengths and styles, and strengthening one's learning and presentation 
skills. EPT members felt that, though the problem-solving classes were generally effective 
in helping groups to be able to work together to deal with common problems, there 
needed to be additional opportunities for individuals to deal with specific "basic skills" 
needs outside the group. 

Through individual counseling, teachers encouraged those learners to take 
advantage of the other classes, tutorials, and computer-assisted learning opportunities 
offered by the company. These forms of instruction were often carried out by the same 
RCS instructors who had taught the problem-solving classes. 

In addition to these more general basic skills and learning-to-learn activities, the 
EPT found a need for more contextualized instruction in such areas as math 
(measurements and quality calculations for the specialty business unit), and writing and oral 
communication skills (for the peer appraisal process). 

RCS has, through its experience at Kodak and other workplaces, developed a 
battery of instructional practices. These include project-based instruction; sensitivity to 
learning styles in lesson plans; and curricula contextualized either to a business as a whole 
or to a particular department of that company. 



Problem-solving at Delphi 

In mid-1996, Rochester City Schools (RCS) instructor Tom Wager joined the EPT at the 
Delphi plant. His task was to develop problem-posing classes which would follow up on 
statistical process control math classes conducted there during the previous year by RCS 
instructor Gina Porter. (See * Statistical process control at Delphi" below.) 

In July, Tom spent time interviewing individual workers to "obtain baseline information 
(about what they did on the job and problems they faced), ensure they had an 
understanding of the objectives of the next phase of instruction (i.e., the problem-posing 
classes), and to develop rapport. " By July 22nd, classes were underway. As Tom 
explains it: 

"We began by focusing on learning styles to develop an appreciation for the 
potential diversity of the team, and some basic ideas for functioning of a team and 
problem solving. 

"During August, we worked on problem identification and problem analysis. 
Management had assigned the problem, but the teams had to develop an 
understanding of the problem by deciding who could help give them insight and what 
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data were needed (to understand the problem more fully). 

" The team had the challenge of dealing with a very complex, long-standing problem 
that had baffled engineers and already had significant resources assigned to it. The 
team wanted to figure out what was causing a leak in a meter body which was 
causing a high number of product rejects. 

"They worked diligently to grasp the situation so they could have some impact and 
produce a potential solution. In addition to solving this particular problem, it was 
hoped that the class would help problem-solving teams to begin regularly to process 
day-to-day problems. 

"As the team got underway, a problem emerged: Due to outside factors, the team 
membership was undergoing change and was being reduced in size. Nonetheless, the 
core that remained undauntedly pursued their assignment. 

"The biggest challenge was that the team's problem seemed to trace back to the 
system and people themselves. The problem itself was not too technical. 

Management encouraged them to come up with non-traditbnal solutions. That 
seemed to free them up in dealing with their assignment. 

"Creating and delivering the midpoint presentation occupied the end of August. The 
participants did an excellent job and management seemed pleased. One worker 
asked a manager what he thought of the presentation. The manager said the 
presentation was beyond his expectations. 

"The team progressed into brainstorming solutions to their problem immediately after 
Labor Day. The problem they were working on had been brought to the company's 
attention by a customer's complaint. A process engineer assigned to deal with the 
problem visited the team. He compared the potential solutions which other engineers 
had generated with solutions which the team had brainstormed. He brought five of 
the team's solutions back to engineering for consideration. That attention from the 
engineering department reinforced the team's efforts. The team went on to finish 
brainstorm ing potential solutions, settled on a few that were most workable, and 
created action plans for them. " 

The team thus produced several outcomes: (a) well-thought-out solutions to a problem 
which had nagged the company for some time, (b) team-problem-solving skills which 
participating workers could continue to apply to other workplace problems, and (c) a 
problem-solving course (with objectives, learning activities, a problem-solving methodology) 
which could be used in the company in the future. 



Paper flow at Albany International at Homer 
Oaanizina the first round, gfjegmjng activities 

Once it was clear to the EPT at Albany International in Homer that paperwork 
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would be the initial focal point for learning, management representative Linda Holland and 
another company representative developed a 1 'model portfolio. ’ This was a collection of 
properly-completed workplace paperwork. From that portfolio, the EPT then developed 
a criterion sheet describing characteristics/ indicators/examples of properly-completed 
paperwork. 

Site coordinator Paula Hayes then showed the model portfolio (with criterion 
sheets) to the production area employees and asked them to collect samples of their own 
paperwork over a period of several weeks. She gave written directions for this task as a 
way of determining whether the participants can follow written instructions. 

Over a two-week period, learners gathered samples of their work in working 
portfolios. This process proved to be a bit tedious, at it required workers to carefully 
collect, copy, use, and store 14 pieces of paper in their folders. Linda and Paula reviewed 
the documents to determine whether/how workers perform particular required tasks. 
(Linda and Paula were the only EPT members to see these documents, to ensure 
confidentiality.) 

This review process was in itself a lot of work -- especially for Linda who was the 
authority on what constituted properly-completed paperwork. Linda reviewed 14 
documents for each of the 30 employees who returned them. 

In some cases, the employees demonstrated their * best work * (their strengths) and 
in some cases they showed the limitations of their current abilities. From these working 
portfolios, the EPT identified training needs to focus on in instructional activities. 

The resulting information was recorded on a specially-designed chart. 

The content one/ formats used, in, ths. initial learning gctiyjtjes 

In the initial round of learning activities, instruction was a combination of one-to- 
one tutorials and group workshops. The former allowed Paula to work with individuals to 
focus on their particular needs and thereby avoid forcing learners to learn things they 
already knew. (The one-to-one format also allowed Paula to fit instruction into the busy 
schedules of workers.) In general, activities focused on the above-described paperwork 
tasks. 



As the workers returned their portfolios in late 1995, the EPT quickly took note of 
particular problems which were apparent in what was turned in. EPT members were able 
to do a quick turn-around and respond to those problems with targeted, customized one- 
to-one instruction carried out in individual conferences. Typically, a learner would be 
shown a better way of carrying out a paperwork task, the learner would go out and try 
it on the floor, and then the learner would return to show the instructor how she or he did 
and to get additional guidance. 

Because several workers appeared to be having trouble filling out accident 
reports, the EPT decided to organize a workshop to help those workers understand how 
to deal with that common problem. 



O 

ERLC 



30 



The portfolio was thus seen as a focal point for learning activities, rather than 
merely as an “ assessment ' tool tacked on to instruction. Learners used materials from their 
portfolios, revised them, added new ones, and thereby documented what they were 
learning as they proceeded. This process helped learners to clarify their needs for future 
learning activities while also documenting in one place -- in a “mastery ‘ portfolio -- what 
they knew. For example, the learners' portfolios demonstrated their ability to follow 
directions and make choices. In the process, the instructor identified who the ‘ experts’ 
were (i.e., those with particular expertise in various areas) and got them to help in the 
instructional process, as mentors and/or classroom resource persons. 

The group workshops allowed learners with common learning needs to work on 
them together with Paula and Linda. 

Paula explained to the learners that collecting items for the portfolio should not be 
seen as a “test’ or threat. She said that, if these assessment activities appeared to 
uncover “gaps’ in learners' knowledge, these gaps shouldn't be seen so much as personal 
flaws as indicators that they weren't given adequate training in the past. Despite these 
reassurances, however, a few workers appeared to be reluctant to return their 
completed documents. 

Paula felt that this mix of one-to-one instruction and short, focused workshops is a 
departure from more-common workplace education approaches which try to “fit’ busy 
companies and workers to traditional classroom formats and schedules. 

This process also created an atmosphere for ongoing career development and 

learning. 

Subsequent learning, activities 

After the paperflow course ended, Paula focused on designing a different version 
of the portfolio. This new version was a checklist containing steps required for various 
procedures or tasks. It was designed to enable workers to use with each other. 

For example, one operator could use the checklist to observe how another, less- 
experienced worker carried out the steps of a particular job procedure. When the steps 
were done, the observer checked them off on the list. 

Paula developed these kinds of check-lists for several of the 45 training modules 
which the company wanted to carry out with all extrusion equipment operators. 

She saw this as a tool to help workers develop a number of SCANS 
competencies, including observation and documentation skills, understanding systems, and 
inter-worker communication and problem-solving. It is a text-based tool to help people 
assess their own needs and those of co-workers and, ultimately, to do their jobs better. 
This is a different mode of helping workers develop these skills than a more traditional 
classroom format. 
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To develop such an approach to learning requires involvement of one or more 
technical experts, perhaps including experienced operators. 

Resources required to develop learning gctjyjtjes 

To enable Paula to do this work, the company gave her an office and computer 
to use. She feels that this intensive exposure to the company -- and her close work with 
company representatives •• allowed her to better understand the company and 
otherwise integrate the basic skills-related activities with other technical training activities. 

Paula states that the company's invitation for her to expand her role there was an 
indication that the company felt comfortable with her and saw the value of investing more 
of the company's own resources in an integrated education-and-training effort. She says 
that this recognition is due to the fact that she spent almost a full year on-site getting to 
know stakeholders and their needs. 

She feels that her experience at the plant has allowed her to develop a process 
for program development and needs assessment which can be transferrable not only to 
other training needs at this site but to other sites as well. 



Statistical process control at Delphi 

The process usee/ to develop the curriculum 

Rochester City Schools (RCS) staff had already had a history of working in the 
Delphi auto plant. Under this new, federal grant, RCS staff worked with an existing team 
which had been set up to oversee the plant's transition to a new product line. In early 
1995, the team did a modified workplace needs assessment, reviewing the strategic plan 
which the team had already developed, to clarify where the CLCI problem-solving 
curriculum might fit in. 

Because statistical process control (SPC) was a clear concern, math instructor Gina 
Porter devoted special attention to finding ways to make math instruction relevant. She 
worked with managers, a quality analyst, engineers, and others on the shop floor to 
review what kinds of math and communications functions were of particular concern. She 
found that SPC was needed to help employees keep track of output from the new, 
precision machines being used to create parts for the fuel injection systems. 

Each machine produces statistics charted on a graph, which show the numbers of parts 
produced and measures of the quality of the product. Workers need to be able to read 
those graphs; understand the mean, medium, mode, and range of the statistics; and make 
decisions based on that understanding. 

Gina developed a focused workshop to help workers understand those math 
functions. They would then practice working in groups to make decisions about whether 
and how corrective action was needed. 
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The content and formats used in the curriculum 

In the second half of 1995, three groups of learners met in classes of 
approximately ten workers. Most participants were motivated to "clear out the cobwebs 
and try their hands at" upgrading their skills in this area, which they saw as important. 

In a typical class, participants worked on their own and in small groups. They focused on 
the vocabulary, concepts, and math functions contained in the charts. This understanding 
was necessary to enable them to talkwith engineers, to diagnose and solve problems. 
Learners also developed self-esteem and personal motivation, oral communications, and 
information-finding skills. All of these are important for ISO 9000 certification as well as 
for day-to-day quality operations and preparation for further training. 

Gina felt that all levels of management (not only higher-level managers but 
supervisors, as well) needed to be receptive to the increased decision-making and 
involvement of shop floor workers. This would require changes of attitude at the 
management level. This is not always easy, given the busy schedules and multiple 
concerns of managers and historic relationships and dynamics between shop floor workers 
and supervisors. 

Elements o{_S. typical lesson plan 

The SPC math curriculum (called "SPC Training ') consisted of two three-hour 

sessions: 

Day / ; Learners focused on statistics on the MetriStat charts used on the floor. 

Qgy 2 : Learners focused on team-building and problem-solving, using simulations of a 
PEP team situation to practice making decisions and solving problems shown on the 
MetriStat charts. (For example, they were given a chart with data in the red zone. 
They needed to analyze what the problem was and then decide what to do, using 
flow charts which showed what steps to follow to solve particular problems.) 

The problem-solving exercises proved to be very popular and were seen as 
relevant by learners. Learners became better at understanding and applying some of the 
team problem-solving training they had received previously, some of which might have 
been too complicated, with too many steps. 

Gina relied on input from the EPT to help her develop the flow chart used in these 
activities. Learners also gave her feedback on the chart, and she revised it as learners got 
more experience using it. 

Gina used this curriculum three times, revising it after each iteration. She 
eventually felt she could use it elsewhere, incorporating different charts and data. She 
acknowledged that plants still geared to more-traditional ways of solving problems might 
not feel comfortable with this curriculum. In those traditional settings, workers might 
expect that their supervisors will make the decisions and therefore see no relevance in 
learning how to solve problems themselves. She felt, however, that as higher-level 
managers go through team training themselves and ISO certification requires more shop- 
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level decisbn-making, these traditbnal ways of making decisions will inevitably change. 



Other support activities 

Gina also oversaw activities at the plant's skills center, which has computer-assisted 
activities and other services. In that capacity, she provided informal tutoring and 
counseling to workers. These additional supports supplemented what she did in her math 
courses. 



Improving communications at E.G.&G. Wright Products 
Tbs, process usee/ to develop the initial cumcujum 

In the initial curriculum development stage in 1995, E.G.&G. Wright site 
coordinator Dianne Spang sat in on company team training sessions conducted by an 
outside corporate trainer. 

To further clarify learning needs, she subsequently met with groups of learners and 
asked each group to do an illustratbn which portrayed their workplace. One group 
portrayed the company as an aircraft named the ‘ SS Wright Products, * which was flying 
high but having to evade various kinds of 1 missiles 8 (corporate challenges) which were 
trying to shoot it down. Another group portrayed a mouse in a maze, trying to find its 
way through unknowns and other challenges. 

In another activity, learners were asked to bring three questions back to the 
workplace, asking co-workers about work-related needs, goals, and obstacles. 

Through such interactions with learners, Dianne identified areas where 
productivity and communbatbns are blocked and possible areas for process improvement. 
From these investigatbns, Dianne identifbd a number of possible goals and projects for 
each class. The groups also showed that there are various ways of interpreting the 
workplace and needed improvements, and that there is a value in getting input from a 
variety of perspectives and individuals. 

TM content and formats used in the 1995 curriculum 

Dianne organized learners into four groups, each with an average of eight 
participants. In each group she attempted to mix employees from different departments 
(e.g., engineering, drafting, assembly, machine shop, testing, and final inspection/shipping) 
and levels (upper level ‘ leaders , * floor managers, and floor workers). 

Each group focused on a different workplace issue or problem which it had 
identifbd in the initial needs assessment phase. The issues were: 

U How to develop a "mother board" system for tracking where work orders are in the 
production process. 

0 How to set up gn on-site training system and facility (using in-house experts as 
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trainers, using actual job-related problems and products in the training, etc.) 



0 How to clarify the skill requirements of each job category and then develop a crossz 
training system to enable employees to prepare for other jobs. 

0 How to develop a problem -tracking sheet to enable employees to analyze causes of 
defects and re-works and to avoid such problems in the future. 

The four groups met an average of seven times each in two-hour sessions, twice a 
week for a month in late 1995. 

Each group went through a variation of a problem-solving activity to deal with the 
issue it had agreed to focus on. They identified "glitches’ in the system and developed 
plans for the above kinds of actions to counter those problems. 

In the process of preparing their plans, learners developed various kinds of 
research (e.g., information-gathering and -organizing) and presentation (oral and written 
language and math ) skills. Although skill levels and expertise varied among team 
members, more-advanced learners tended to resist dominating the team process, and less- 
confident learners were encouraged to "stretch" themselves. 

As they developed their action plans, each participant was expected to go out on 
the floor and present the team's draft recommendations to three co-workers. Learners 
noted the co-workers' responses and brought them back to the group. The group in turn 
incorporated those responses into their final presentations. 

Participants were encouraged to decide what roles they would play when their 
groups presented their action plans to the plant leadership team. (Most made an oral 
presentation, but one participant preferred to organize the slides on the overhead 
projector.) Each group member was asked to prepare a "script" which he/she could refer 
to during the presentation. 

The groups made their presentations in late 1995. The leadership team considered 
the plans for four weeks, then asked the groups to be more specific about costs of their 
recommended actions. Dianne subsequently worked with some of the groups to come up 
with budgets. 



A communications course at Albany International in East Greenbush 

The content gruj formats proposed for the initial phase o£ the curriculum 

Based on the WNA conducted by the Albany International EPT at East 
Greenbush, site coordinator Margaret Shirk anticipated that the content of the first 
round of instruction would be a 15-hour introductory ("foundation") course titled * Skills in 
Context: Communication and Knowledge Systems at East Greenbush. " Here is Margaret 's 
description of the course: 
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* The objectives of this course focus on developing the following skills and 
knowledge: 

communication skills for speaking and participating in a variety of situations 
including problem solving, decision making, future training, and day to day 
questions and issues. 

strategies for the organization and oral presentation of information. 

learning skills specific to individual learning from written directions and manuals 
(technical reading), oral instructions and presentations in formal learning contexts 
such as classrooms, meetings, and committees, and on-the-job training. 

“ The goal of this course is to develop the skills and strategies needed for employee 
participation in solving job problems, improving job-specific skills, and planning for 
future education and training for the continuous improvement of work-related and 
individual abilities and skills. By focusing on communication and learning skills, this 
course will provide a context to develop a positive relationship between the human 
and technical systems that employees participate in here. 

“ This course will develop an understanding of work and education in America, with 
an explicit emphasis on developing learning skills and communication skills. It will serve 
as the foundation for all subsequent courses. These skills and knowledge also establish 
a context for better understanding ' the big picture' that shapes the actions and 
fortunes of individual companies and their employees. " 

The course was to be "problem -posing" in the sense that it would provide a 
framework within which employees would be able to identify specific issues, concerns, 
obstacles, and goals around which they want to focus learning activities. Margaret 
anticipated six different learning groups, all using the same framework but with different 
specific issues (e.g., communicating with supervisors and co-workers, preparing quarterly 
reports) unique to each group. 

This process of customization (i.e., working out learning goals, etc.) would in itself 
develop participants' language skills and abilities to identify needs and problems. This 
course would fit the culture of the plant because all employees had gone through TQM 
training and understood this kind of lingo. ’ 

In addition to focusing on workplace-specific issues, the course was to focus on 
learning and education. It would try to get participants to to reflect on school and other 
prior learning experiences, to put workplace learning in a different light and make their 
assumptions about learning more explicit. 

Margaret also wanted participants to write a job description and use it as 
something to add to and elaborate on over the course of the program. This would help 
them look at their own work and identify how they use SCANS skills. 

To develop listening and speaking skills, learners were to do such things as 
"communication mapping," identifying how communications and decisions were carried out, 
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obstacles to good communication i, and ways to improve communications. They might also 
do role plays which they could observe and analyze and in which they could develop 
various communications skills. 

Drawing on her previous experience as a writing instructor (especially for ' non - 
traditional' college students), Margaret wanted to set up a class environment in which 
everyone was expected to speak and write. She would have learners do writing that 
wouldn't be collected but which would get them into the mode of thinking of themselves 
as people with something to communicate. These writings could, in turn, be used as notes 
for further discussion. 

This expectation was established as a way of overcoming 1 the reluctance factor ' 
often found in basic education settings. She felt it would be important for the instructor 
and learners to be clear about the goals they are aiming at in the group. 

In addition to communications, the initial classes would focus on * understanding the 
big picture' of the contexts (social, economic, political, and educational) in which learners 
and the organization operated. This would be done through readings taken from daily 
newspapers, the plant newsletter, the corporate newsletter, and elsewhere. (She felt 
that the plant newsletter used language and covered topics of more-direct relevance to 
most workers than did glossier publications.) 

While the above curriculum was still in a fledgling stage, Margaret anticipated that 
it might eventually be developed into a generic model for an * introduction-to-workplace - 
education- and-change' course. She was interested in learning from others who have set 
up similar courses. 

What the curriculum actually looked, like 

In a report written by Margaret in December 1996, she noted that in that year 
“ almost every hourly and salaried employee in the plant's manufacturing section 
participated in a 15-hour communications course. " This course “ focused on the continuous 
improvement of both the technical and social systems of felt production" at the plant. 

“ The communication skills developed in the class included listening, speaking and 
participating in a group and in one-to-one exchanges, understanding others and 
addressing differences constructively, responding to maintain positive relationships, 
asking questions, and various strategies for securing better understanding between 
peers and in relationships involving responsibility and authority differences. In 
short, the course functioned as a communication problem-solving workshop in 
which all members participated and contributed. " 

Participants identified and then wrote about instances in which communications 
among employees were confused or otherwise not effective. They analyzed the causes 
of communication gaps and suggested ways to improve them. The participants' writings 
were assembled into readings which others within the class and plant could reflect on and 
learn from. 
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Participants uncovered and discussed obstacles to communications which were 
both individual and organizational in nature. The coordinator concluded that . . . 

. . developing the skills of individuals is only part of the corrective change 
necessary to improve communication . . . Successful improvement of these 
problems depends on addressing these issues from an organizational perspective as 
well. Good business practice > and common sense, tell us that quality and e fficient 
production depend on both the technical and social systems of an organization 
functioning together to create a consistently high quality product and a working 
environment that effectively promotes the day to day achievement of this goal. 

The education program in general was described by many of the people I spoke 
with as an 'opportunity. fo£ everyone to get on the some eoge here. 

Margaret reported that, although the participants had previously undergone 
training in TQM practices and procedures and a number of new procedures and 
standards had been introduced in the plant, not everyone understood these changes fully 
or communicated as effectively as possible. She says that "the completion of these classes 
provides an opportunity to address the issue of standards and procedures, and to make 
current policy and practice clear to everyone. Policy and procedural practices are not 
consistently viewed by employees here, resulting in communication problems that cause a 
loss of time, focus, and good will . " These problems were related both to technical 
"production procedures" and to social ("people") policies such as time off and benefits. 

She felt that the classes provided opportunities for all employees to help develop 
and understand the “ explicit procedures and consistent practice" which serve as "the 
foundation for effective communication in an organization. “ "If policies are not clearly 
defined and communicated, and then consistently maintained, the consequence is time lost 
to communication problems . " 

The class helped to air and rectify what employees saw as "double standards": the 
gaps between (a) written standards and procedures and what is day-to-day common 
practice and (b) high-quality standards and day-today production quotas. 

The coordinator concluded that ” People here want to do their jobs well. Everyone 
wants to workwith initiative and good will. Many of the course evaluations stressed the 
need for the knowledge and skills learned in the communication course to become 
organizational practice. " 



Understanding company benefits at Albany International in Menands 

Tb§. process used, to develop the cumcujum 

At the Albany International plant in Menands, site coordinator Judy Lees went 
through a multi-stage process of getting input from the EPT and then learners themselves. 
This work built on the foundation of the company's Learning Enhancement Program (LEP), 
which had begun with individual basic skills assessments in late 1993 followed by basic skills 
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classes in 1994. To supplement information gathered and lessons learned during the LEP, 
the coordinator collected information via (1) a WNA (which relied on focus groups, 
individual interviews, and document reviews), (2) initial input from a group of learners 
organized in a ' problem solving" course, and (3) subsequent, ongoing input from those 
learners once the course got underway. 

The content ond formats used, in the problem-posing curriculum 

While the initial round of instruction focused on helping people to use the 
company's new benefits package, using the package was not itself the end goal of 
instruction. Rather, learners were to improve their abilities to gather information, manage 
it, analyze it, and make decisions with it, both as individuals and in a team setting. 

These information-processing, planning, and decision-making skills were seen as applicable 
to many problems/questions workers face on their jobs, and not just to "using the benefits 
package. " 

Judy set up each class as a TIT" (process improvement team) which emulated the 
teams they worked in back on the floor. In a 40-hour course, the PIT examined the 
problem of how to get essential pension-related information to interested employees so 
they can use it. " The PIT felt that the company's previous method of merely ” handing 
employees a book" was not enough. As an alternative, the team figured out a way to 
present the information via a seminar. They implemented the seminar and attracted many 
employees who came after work hours. As a sign of the positive impact of the course, it 
was at employees' request repeated twice for other employees. Also, at the end of the 
course when workers were asked to list in their learning logs what they had learned, 
several stated that they were now far better able to read the technical material they 
were given. Some also said that the math they had learned in class made them more 
appreciative of the value of the benefits package offered by their employer . 

Problem-posing regarding training opportunities 

Judy conducted the above "benefits-package" course with workers from the first 
and second shift. She took a different focus for the work she did with the third shift. 

Here is how Judy described the work she did with the third shift workers: 

" During third shift, the 'Process Improvement Team' (i.e., the class) decided to deal 
with the issues of training opportunities. The segment of workers who were most 
disenfranchised were the pinseamers. Pinseaming is 'women's work' according to the 
floor workers and the H.R. staff, and the pinseamers on all shifts are referred to as 
'the girls.' The work requires fine motor skills and extended concentration. 

"Because the research engineers and designers all work days, the pinseamers who 
work first and second shift have the opportunity to work with the new fabrics and 
seams. Most often these interesting and challenging fabrics are completed during the 
first and second shift when the engineers and designers are there for a portion of the 
shift to explain and oversee the work. As a result, the first and second shift 
pinseamers do more interesting work, have more contact with management, develop 
increasingly diverse skills and understandings, and increase their self-esteem. 
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"This increased self-esteem, unfortunately, feeds a sense of superiority to the third 
shift 'girls' who resent the double whammy of being denied an opportunity for 
improvement, and then being 'dissed' for their lack of versatility. 

"Using the problem solving curriculum, our 'Process Improvement Team' (i.e., the 
class I submitted a request to management that resulted in a designer going in two 
hours early for a month to train selected pinseamers on some of the more innovative 
fabrics and seams. 

"Since that time, the machinery is becoming increasingly computerized, and it is 
economically necessary for the employer to utilize the complex equipment 24 hours a 
day. Therefore, the third shift pinseamers are being exposed to increasingly complex 
technology. 

"A significant non-training benefit of this designer's regular training visits to third 
shift was the impact on the workers of management 's caring enough to consider AND 
ACCEPT their proposal. Throughout the problem-solving process, the participants 
repeatedly, heatedly insisted that the 'suits' didn't care and wouldn't do anything. 
Management's acceptance, consideration, and implementation of the third shift class' 
proposal was a morale booster through all departments on the shift." 

Judy felt that, although her work with the third shift workers produced positive 
results, she might have been even more successful if she had been more aware of the 
particular mentality of workers on that shift. "I'm sure I would have accomplished more if I 
had had some special training before I went in to third shift!" She also felt that workers 
would have been more motivated and encouraged to participate if management 
representatives came to visit the classes. 

Given the above experience with workers on all three shifts, Judy feels that 
workers at Menands would benefit from further work on "communications, effective 
listening, and non-confrontational conflict resolution. " 



Introduction to computers at E.G.&G. Wright Products 

In spring 1996, E.G.&G. Wright site coordinator Dianne Spang conducted a 
computer needs survey of all levels of employees, to have them assess their own abilities 
and needs vis-a-vis computers. The company wanted everyone to be "computer literate" 
as soon as possible, so that all employees would be able to use the new computer system it 
was investing in for the plant. (To reduce paperwork, all workers would need to be able 
to use e-mail and to fill out human resource department forms -- like vacation requests - 
on a computer.) To use this new equipment, workers needed to know how to use the 
Windows program, so the survey focused in particular on identifying those who needed 
"pre-Windows" training. 

When the survey was completed, the EPT looked at the results and decided what 
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the cut-off points were for various levels. (For example, an entry-level course might focus 
on basic keyboarding; the next level would be an overview of the MicroSoft Office 
program.) Based on the survey results, the coordinator and another instructor developed 
a curriculum for those various levels. 

The result was a series of six 15-hour classes for groups composed of an average 
of ten representatives of various departments and job categories. Classes were 
voluntary, although the coordinator sent notes to particular individuals encouraging them 
to participate if the survey indicated particular needs. 

Employees signed up to participate, and the EPT decided who would come to 
which class, to ensure that there wouldn't be too many employees from one department 
participating in any one class (and thereby hurting that department s operations). 

The resulting mix of employees in each class proved to be useful, as it reinforced 
the cross-department communications which the original "communications" course had 
fostered. Participants talked about the computer classes between sessions, and those 
with more computer expertise helped those who had less. 

Participants practiced using what they learned in class back on the job and at 
home. One participant, for example, learned how to better understand the home 
computer she had bought for her family. She in turn became a mentor to fellow workers 
back in the plant. 

The coordinator felt that this 'back-and-forth" between the classroom and practice 
outside the classroom was very useful, given the limited practice time that a 15-hour 
course could provide. The natural mentoring which occurred led to the creation of a list 
of in-house “ computer consultants" who were willing to share their expertise with co- 
workers. 

The computer classes were offered first in the spring of 1996 and once again in 
the fall of that year. Because there was a delay in purchasing and installing the 
company's new computer system, the EPT agreed to hold some additional one-day 
"refresher" courses when the equipment was installed. The company appears willing to 
pay for these courses even after the federal grant runs out, because it recognizes the 
need for and value of such activities. 

The coordinator has a written version of the curriculum used for this computer 

course. 



Shop math at E.G.&G. Wright Products and Albany International in Menands 
EG.&G. Wriaht Products 

In the original WNA conducted at E.G.&G. Wright, the EPT had identified the 
need to upgrade employees ' math skills, especially those needed to handle tasks related to 
"geometric dimensioning and tolerandng" functions used with tools and parts. An 
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instructor conducted a math skills assessment which focused on math skills used in the 
plant. 



The resulting classes used the same kind of problem- posing mode used in the 
above-described communications and computer courses, asking learners to identify work- 
related math problems and then focusing learning activities around those problems. 

The math course lent itself to generating " numbers " (quantitative data) about 
learner abilities ("You either get the right answer or you don't. ") These "numbers" could 
be summarized in each learners' portfolio. 

Albany IjTtemgtjgngl at Menands 

In late 1993 through 1996, the Menands program turned to math as its primary 
focus. College math-and-business instructor Bill Eckert was hired to run a series of 
mathematics courses which covered such topics as algebra, trigonometry, geometry, 
probablility and statistics, use of the scientific calculator, charts, conversions, and personal 
and corporate finance. Classes were divided into three 10-week sessions consisting of one 
2-3-hour class each week. 

The instructor encouraged learners to find relevant tasks from their daily lives on 
and off the job to focus instruction on. For example, learners might review the math 
required to understand their company benefits package or how to build a deck. 

To assess and track learner skills and needs. Bill would conduct periodic ” teacher- 
made " tests. He would share results with the learners but not with management, to clarify 
how learners were doing. He would give general but not worker-specific feedback to 
management, summarizing in general terms what learners were learning and feedback 
which learners gave on an end-of-session evaluation form. 

Learners said that the courses were "interesting," enabled them to help their 
children with schoolwork, helped them better understand job-related math, inspired them 
to go back to college, and helped them with home projects. 

Lessons learned about the curriculum model 

The six sites which implemented learning activities produced a wealth of tools and lessons other 
workplace educators can learn from. One important *- perhaps the most important *- lesson 
learned in the CLCI is that t here are many learning needs which a workplace basic skills effort can 
focus on . More specifically: 

Each sjte identified Q number of problems to focus its basic skills program on . These ranged 
from "improving communications" to "introducing workers to computers" to "shop math" to 
“ understanding benefits packages. " No single set of lesson plans will be relevant for all 
workers, regardless of their background or the contexts they work in. 

It is important for, an EEL t_o have the flexibility -- and a comprehensive, "open-minded" 
gssessrnent process -• to be able to identify the range of learning needs represented in a 
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particular workplace. 

In turn, education staff need the expertise time, grid other resources to be able to put 
together a package (or "battery") of basic-skills-related services which are meaningful and 
effective in responding to the learning needs which continually evolve in a typical, dynamic 
workplace. 

We also learned that a basic skills program which focuses on"problem- posing" and "problem- 
solving" does appeal to a variety of types of companies and employees . However, that concept 
can be interpreted in a variety of ways, as shown in our examples above. The key is to 
recognize that adult basic education can help people become active thinkers and problem- 
solvers, and that they can use those skills at work and in other life contexts. At the workplace, it 
is important for others in the organization -- managers, supervisors, union representatives, and 
co-workers -- to support learners when they try to use their problem-solving skills back at work. 



CHAPTER 5 



Portfolio Assessment 



Each educational planning team (EPT) was expected to design and use assessment 
activities to identify learner needs, abilities, and interests, and to track learner progress. Sites 
were asked to use portfolio assessment techniques, and the resulting information was to be used 
by the learner and instructor to encourage and guide the learner. Learners' assessment 
information could also be summarized in an aggregate, anonymous form and presented to the EPT 
and other audiences interested in understanding what was going on. 

This chapter (a) summarizes the assessment-related guidelines presented to the sites, (b) 
describes how the sites interpreted those guidelines, and (c) discusses some "lessons learned'' about 
using portfolio assessment in workplace education. 



Guidelines for using portfolio assessment 

Individual assessment is, in the CLCI model, not something that occurs only at the 
beginning and/or end of an instructional period. Rather, it is seen as a tool to involve the 
learner and teacher in an ongoing, thoughtful dialogue about the learner's needs, abilities, 
interests, and progress. It also can be linked to other program components, especially 
workplace needs assessment, curriculum, and evaluation. 

How "assessment" has been viewed to date 

Until recently in the workplace education field, the term "assessment" has tended to be 
associated with ... 

/. a test of some kind (either a standardized test or one customized to the particular 
demands of participants' jobs), and/or 

2. a Ijteracv task analysis, a procedure for determining the literacy skills required for 
particular jobs and, subsequently, whether workers in fact possess those skills. Such an 
analysis would clarify for curriculum developers who needed to learn what to do their jobs 

better. 

Assessment from a collaborative perspective 

The CLCI model emphasizes collaborative decision-making as an outcome of -- and process 
for -- workplace basic education. We proposed that assessment follow these guidelines: 

0 Assessment should be seen gs one ofsevergL decision-making tools which stakeholders 
can use to clarify learning goals, map out strategies for meeting those goals, and monitor 
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and refine those strategies. Assessment should thus focus less on identifying deficits and 
more on clarifying goals and abilities (resources, capacities) of the organization and 
individuals involved, and on mapping out strategies for using existing resources (knowledge 
and skills) - and building on and further strengthening them -- to enable the organization 
and individuals to improve themselves. 

0 As such, assessment has much in common with the monitoring tools used in "total quality 
management " and related approaches to organizational development which emphasize 
continuous planning and improvement of operations to meet customer needs. 

0 Because workplace education needs to focus on changing both individuals and 
organizations, assessment likewise should focus on both individuals and the larger 
organization. Assessment might thus be divided into ' individual assessment" and 
"workplace (organizational) assessment" procedures. 

0 Assessment procedures need, to focus on information which stakeholders really need, to 
make decisions, rather than on less-relevant information which distracts stakeholders from 
what they are really trying to accomplish in their educational efforts. 

0 Assessment procedures should be user-friendlv and make it easy for stakeholders to get 
involved in program decision-making. 

0 Individual and_ workplace needs gssessrnents should be integrated with each other and 
with related functions of program evaluation, curriculum development, and staff 
development. 

0 For many individuals in a workplace, assessment is often associated with "tests, " 
"performance appraisals, " and similar situations in which individuals are judged or 
evaluated. Assessment is thus often seen as a potential threat. Assessment procedures 
must therefore be presented to learners and other stakeholders as a positive decjsjgnz 
nTakjng tool • Those involved must maintain the confidentiality of what is said and avoid 
using it to punish or in any way jeopardize those involved. 

Portfolio as a container for various kinds of assessment procedures and information 

We proposed that a portfolio should not be seen as a rigid, prescribed assessment 
procedure. Rather it should be viewed as a vehicle for collecting and using information about 
learner needs, interests, abilities, and progress. This information should, in turn, be used by 
stakeholders -- learners, facilitators, and others -- to shape learning activities so that agreed- 
upon goals are met. 

What information did we want EPTs to collect? 

Individual assessment has many of the same purposes and procedures associated with 
workplace needs assessment and program evaluation. All three are mechanisms for collecting 
some kind of information which, in turn, someone will use to make some kind of decision. 

When planning any of these three decision-making functions, it is important to be clear 
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about what kind of information will be collected. Sites were urged to use portfolios to 
document how learners dealt with the various kinds of SCANS tasks they needed in their 
jobs. 

While we pointed them to various references about portfolios and workplace assessment 
and evaluation (See the References in Chapter 8.), sites were generally left on their own to 
determine what to include in their portfolios, and how to gather, organize, and use that 
information. 



How the sites interpreted portfolio assessment 

The Central Planning Team provided the EPTs with above kinds of guidelines as well as 
various handbooks and some professional development workshops, to help them figure out 
how they might use portfolio assessment methods in their sites. In practice, the sites paid 
various amounts of attention to assessment and produced several different types of 
assessment activities. 

Most sites made some attempt to use a portfolio approach, as described below: 



0 At the Albany International plant in Homer, portfolios were the medium for both 
assessment and instruction. As described in Chapter 4 , coordinator Paula Hayes took 
a creative approach to portfolio assessment which blurred the line between 
assessment and learning as the portfolio became a vehicle for learning rather than just 
a means to think about or assess learning. Workers used portfolios to collect samples 
of the paperwork they did on the job, work with instructors to clarify the strengths 
and limitations of that paperwork, and then get individualized or group instruction and 
practice to make needed improvements. This process helped keep learning activities 
focused on priority learning needs and produced clear documentation of learner needs 
and progress. See pages 29-32, “ Paper flow at Albany International at Homer, " for 
more details. 

Paula concluded that this use of portfolios is particularly useful in a workplace 
education initiative where procedures and operations are the focal point. This 
process also is helpful to a company which does not have written procedures in place. 
A company can workwith workers to develop written procedures, help them master 
those procedures, and then have a written record of what each worker learned. This 
is especially important for companies undergoing IS09000 certification. 



0 At the Albany, International pjant in Menands. coordinator Judy Lees tried a number 
of assessment tools, with varying degrees of success. She wanted to be able to 
document learners' abilities to understand and use the company's financial benefits 
system. She summarized this experimentation as follows: 

"In terms of reading/writing/math, the worksheets I made up tied to various pages 
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in the retirement handbook and medical coverage manual were the best I could do. 
However , to keep learning non-threatening, so that I didn't lose participants, I 
found it beneficial to allow workers to collaborate. . . . Coming from the American 
public school mentality, I find it uncomfortable not to have a pre- and post- 
assessment for each individual. However, the reality is that these individuals work 
as teams in their workplace, and the ability to participate in effective collaboration 
is of sufficient value that I guess I've become willing to have only a pre-post 
assessment of the team s abilities (rather than of individual abilities). " 

After the first round of instruction, Judy said that she had questions about how to 
document the impact of the program on the workplace itself. This is especially difficult 
because many of the skills being focused on are not easy to quantify. Another 
problem with measuring impact is the fact that supervisors and others who might be 
able to give feedback about worker performance are busy with other duties and don't 
have the time to observe and reflect on subtle changes in worker * communication , * 
"problem-solving," and "decision-making" skills. 

Beyond measuring transfer of learning is the question of how to ensure or facilitate 
such transfer. In other words, it is not enough to try to determine whether learners 
are using what they'e learned back on the job; it also necessary for workplace 
educators to figure out what needs to be done to help workers effectively apply 
what they have learned. 

Judy also wants to be sure that whatever is measured in assessment activities is in 
fact what is taught in her courses. In other words, it is not fair to teach one thing and 
then judge the effectiveness of the course by measuring something else. 



0 For a math-for-statistical-process-control course gt the Delphi plant in Rochester, 
instructor Gina Porter asked each participant to keep a folder with examples of 
his/her work. She later acknowledged, however, that this course was only a six-hour 
course and could therefore produce only a limited amount of evidence. 

As she developed a portfolio system, she wasn't sure whether the portfolios 
should be oriented to showing mastery or progress or both. (Mastery, she felt, is 
useful for accountability and motivation for both individuals and the company.) 

Once Gina received the learners' portfolios, she reviewed them, using rubrics she 
had developed, to clarify the extent to which they had mastered what was being 
taught. Her rubrics were developed with input from EPT members. To create the 
rubrics, she identified several learning objectives and then noted possible indicators 
which would show whether those objectives were met. 

After Gina came Tom Wager, who was to run a problem-solving course in 
another section of the plant. Tom worked with the external evaluator, Don Gchon, 
to develop several assessment instruments to document what learners had mastered. 

In one assessment activity, learners were asked to identify steps in the problem-solving 
model, name and describe the purpose and mechanics of five problem-solving tools, 
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and define the difference between consensus and compromise. Learners were also 
asked to rate themselves in terms of five SCANS skills and to also rate the course and 
instructor. The resulting assessment and evaluation data were compiled into a " site 
portfolio" for Delphi. 

Tom also noted a number of anecdotes which served as evidence of positive 
program impact. One woman, for example, said that as a result of participating in the 
problem-solving classes, she was now more involved in committees at work and in 
similar committees in her church. She said that, prior to the class, she was always 
afraid to speak up and get involved. Now she was more confident and knew how to 
play a constructive role in team situations. 

Tom felt that the most tangible evidence of positive program impact was the 
action plans developed by the teams. These were concrete indicators that the teams 
were in fact now able to carry out the problem-solving process covered in the course. 



0 At the Eastman Kodak facility jn Rochester, instructors did a lot of work to develop 
the use of portfolio assessment. Staff came away convinced of the value of this form 
of assessment. They stated that portfolios . . . 

. . enable us to assess growth in a contextualized workplace format that 
specifically addresses the obiective of the training. They allow us to show 
growth in non-educationally-traditional SCANS skills. They do not assess in 
terms of grade levels, but instead in a way which fits well with capability 
certification tools. They are productive educational tools which allow the 
learner to develop skills in self-evaluation and goal-setting. " 



0 For a workplace communications cours e gt the E.G.&G. Weight giant 
site coordinator Dianne Spang used portfolios for both individuals and for groups. For 
each learner, she gathered artifacts which showed what each learner was producing. 
She also asked them to reflect on the activities and give her feedback. 

She also developed a post-course survey in which individuals clarified (a) whether 
and how they improved various SCANS competencies in the class, (b) whether and 
how they are now using those competencies back on the job, and (c) what impact 
those uses of SCANS have had on the workplace. 

She also pulled information from those individual portfolios to include in a class 
portfolio for each group. That portfolio included the group's action plan. 

Despite her creative use of portfolios, Dianne felt it would have been helpful if in 
the initial stage of the project the Central Planning Team had given her more ideas on 
what to include in the portfolios (especially for the outside evaluation). She felt that 
the site EPT did not ask for highly-quantified data, but that she nonetheless would 
have liked to know how to develop systematic rubrics which might produce such data 
in case anyone wanted it. 
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To tailor the portfolio to an " introduction to computers" course, Dianne worked 
with EPT members to develop rubrics for particular computer-related skills needed in 
the plant. She then referred to those rubrics when talking with individual learners 
about their needs and skills. From these discussions came recommendations for next 
steps which each learner needed to take to improve particular skills. 

For each skill area, the following rubrics were used to indicate the levels the 
learner had for that skill area: 

0 Expert 
0 Proficient 
0 Intermediate 
0 Novice 

The skill areas focused on in the course and in the portfolios included: 

— keyboarding 

-- managing equipment (e.g., turning the equipment on and off...) 

— mouse skills 

-- management of files 

— creating a document 

The above rubrics emerged from an initial computer skills survey and from 
subsequent discussions with EPT members and learners. 

In her discussions with learners, the coordinator not only focused on how learners 
used computers on the job but in their lives at home as well. The company saw this 
focus on " not-strictly-job-related " skills as appropriate, as it recognized that, by using 
their skills at home, learners would be "reaffirming" (i.e., reinforcing) the computer skills 
they were learning at work. These skills would then be transferred back to their 
performance on the job. 

The coordinator saw the portfolio as a focal point for discussions with learners in 
which they could produce sample work over time, to show initial abilities and abilities as 
they changed. The samples would include both "good" and " weaker " work to show 
the range of abilities in various areas, progress, etc. 

The company never pushed the coordinator to use standardized assessments. 
Instead it encouraged her to develop tools which were customized to the particular 
learning needs of the employees and plant. 

Dianne Spang concluded she will likely continue to use portfolios in future 
workplace education programs. She would also like to do a follow-up assessment via 
an on-line survey or other means. In that survey, the coordinator would ask the 
participants (a) whether and how they are using the skills they learned in the courses 
and (b) what else they feel they need help with. 
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Dianne feels that, though useful for learners and instructors, portfolios can 
sometimes seem inscrutible to other audiences like employers. She recommends that 
the evidence and terms used in portfolios be boiled down into language which an 
employer can use quickly, perhaps as part of an employee's professional development 
plan. 



Lessons learned about portfolio assessment 

The educators who used the above forms of portfolio assessment all affirmed that this was 
an appropriate way to generate useful information for learners and other stakeholders. 

However, one down side of using portfolios was the amount of time required to design them (e.g., 
to work with learners and others to figure out what information to store in the portfolio and to 
create rubrics for rating the quality of those artifacts) and then use them properly (i.e., portfolios 
are places for thoughtful reflection and dialogue about learning over time, not merely a vehicle 
for quick snapshots of learner abilities). 

In addition to requiring time of instructors and others who are to use them, portfolios 
require instructors to have special kinds of expertise. Special training, resource materials, and 
supervision need to be provided to anyone attempting to use portfolio assessment in workplace 
settings. 

Finally, those who are interested in going to the effort to use and further develop these 
forms of assessment must keep in mind the limited background and attention span of some 
employers, funders, and others who presumably should be interested in good-quality assessment 
data. It can be frustrating to go to the effort of developing high-quality assessments and then 
find that the decision-making audience the report is prepared for simply doesn't have the 
expertise and/or time to understand and use the information you are providing them. 
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CHAPTER 6 



Program Evaluation 

As described in the preceding chapters, the educational planning teams were responsible 
for conducting workplace needs assessments and designing and carrying out instructional and 
portfolio assessment activities. One other major area of responsibility for EPTs was the planning 
and conducting of program evaluation activities with the assistance of two outside evaluation 
consultants. 

The outside evaluators served as resource persons on the project's Central Planning Team 
and worked with each site to develop an evaluation plan and evaluation tools geared to the 
specific outcomes which the sites were aiming at. The evaluators produced a four-part series of 
evaluation reports which summarize this process of planning and carrying out a customized team 
evaluation and the findings which emerged from the evaluations at each site. (See the References 
in Chapter 8.) 

Here are some highlights of (a) arguments for a "team" approach to evaluation, (b) the 
evaluation process used at each site, (c) project outcomes identified in the site evaluations, and (d) 
lessons learned about the use of "team" evaluation. 

Arguments for a "team” approach to evaluation 



CLCI drew on prior research and development projects which had developed "team-based" or 
"collaborative" approaches to evaluating workplace education programs. (See References in 
Chapter 8.) The proponents of a team approach to evaluation argue that: 

0 Too often, "evaluation" has not been given much priority in workplace education . It has 
often been seen more as a burden or distraction and as something done by an outside 
evaluator who generates marginally-meaningful information for a funder or someone other 
than those actually doing the work. 

0 Those organizing a workplace education effort instead, need, to see evaluation gs a vital 
component a way to continually keep efforts efficiently focused on priority needs and 
objectives, in the spirit of “ continuous improvement. " 

0 It is. jnipprtgnt for a wide range of program stakeholders to be involved, in the evaluation 
process (via an educational planning team, or EPT), to help those stakeholders better 
understand the program, feel ownership for it, and invest in making the program a success. 

0 EPT members gre busy people gnd_ need, an efficient process through which they can 
participate in the planning and implementation of the evaluation. This includes reflecting on 
and making decisions based on the information generated by the evaluation. 

0 EPT niembers need, to decide what kinds of. information they really want the evaluation to 
generate, to ensure relevance and efficiency of the evaluation. In turn, those collecting 
information need to ask the right kinds of questions and use the right kinds of information- 
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gathering techniques. This typically requires custom-designing of interview guides, tests, and 
other evaluation tools. 

0 To do this work, EPT_ members need, time, expertise, gnd a cormiitment to doing hiah-qualitv. 
meaningful evaluation . 

The evaluation process used 

Evaluation activities took place at two levels of the project: at the "statewide" level (done by the 
two evaluation consultants in their interactions with the seven sites) and at the "site" level (done 
by the evaluation consultants in collaboration with the educational planning teams). 

The evaluation consultants wanted to serve the dual role of "external evaluators" (to make sure 
relevant information was prepared for the funder and other outside audiences) and "facilitator of 
internal evaluations" to be conducted by each EPT. 

The consultants spent much of the first year of the project getting to know the sites and working 
with the EPTs to develop site-specific evaluation plans. Each site was to develop a "site 
portfolio" which presented what it hoped to accomplish in its education program, evidence of 
what actually happened, and the process each site went through to get those results. 

In the second and third years of the project, the EPTs prepared reports with and for the 
evaluators, and the evaluators summarized those reports in annual project-wide evaluations. 

In addition to helping each site EPT to design and carry out an evaluation, the evaluation 
consultants worked with the Central Planning Team to clarify goals and indicators of progress for 
the CLCI project as a whole. For example, the CPT wanted to provide an infrastructure (of 
funds, training, and technical assistance) to help the site-level EPTs do the work they wanted to 
do. The CPT also wanted to field test, evaluate, refine, and disseminate "the CLCI model." 

In their four-part evaluation report, the evaluation consultants summarized what happened at 
the levels of both the EPTs and the CPT. 

Project outcomes identified 

The evaluators' final reports indicate that: 

0 CLCI was fairly successful in developing and testing "the CLCI model" described in this 
handbook. 

0 CLCI was fairly successful in providing training and other technical supports to help the sites 
meet their goals. 

0 As the project wound down, it was not so clear to what degree and how the CLCI model 
would be replicated elsewhere after the federal grant ended. While some sites in New York 
State were likely to use elements of the model and CPT members had disseminated the model 
nationally via workshops and publications, the future of workplace education in general and 
the CLCI model in particular were not clear. 
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0 It is difficult to state with much certainty what impact the project had on workplace 
operations in each site , even though the curriculum focused heavily on skills and tasks 
considered high-priority by EPTs. This was in part due to the difficulty of quantifying a direct 
link between education (or training) and "productivity" (productivity being something 
determined by many factors other than worker skills or the education they receive). 
Nonetheless, most sites could cite rich anecdotal information showing positive impact on the 
workplace and workers. 

0 Participation of employers and workers in EPT activities varied from site to site and year to 
year. This was due in part to the many competing demands place on employers and workers. 
It might have also been due to internal factors in the organization's culture (e.g., low priority 
to nurturing employee growth) beyond the control of outside educators. 

Lessons learned about team evaluation 

The above-described evaluation operated at two levels and in seven sites. In each site, a multi- 
stakeholder educational planning team was asked to get involved in designing and carrying out 
evaluation activities. As might be expected in so complex an evaluation, the team evaluation 
model worked better in some sites than in others. 

Overall, however, the response of sites was generally favorable about the evaluation model 
used. However, experience also showed that evaluating workplace education programs beyond 
a superficial level is not easy. Here is a summary of feedback from the sites: 

0 Most site coordinators welcomed the involvement of outside evaluation consultants who 
took a “ facilitator " rather than * inspector " approach to evaluation. 

0 All agreed that the evaluatbn process needed to be kept simple while at the same time 
focused on meaningful, reliable information about program impact and best practices. 

0 The success of the EPTs and the work they did (including their evaluations) varied from site 
to site. Success depended on the leadership and vision present within the team; the climate 
of the company (some sites were undergoing rapid change which prevented people from 
focusing on EPT tasks); how much time EPT members could give to planning, implementing and 
paying attention to an evaluation; and the expertise which EPT members had about 
workplace education and evaluation. 

0 Evaluating outcomes is a challenge in a workplace education program, as it requires all 
concerned to identify reasonable objectives (both at the start of the project and then on on 
ongoing basis), set up a system for monitoring progress toward those objectives, and then 
take the time to monitor progress. In the CLCI project, we adapted the Kirkpatrickmodel of 
workplace training evaluation, and tried to look for several levels or types of outcomes. We 
wanted to know whether . . . 

(a) the learners were mastering the skills and knowledge being taught in their particular site, 

(b) the learners were actually applying the newly-acquired skills and knowledge back in 
their jobs, and 

(c) if so, whether the learners' applications of those skills was having a positive impact on the 
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organization. 

We were trying to get away from the "smiley sheet" approach to evaluation in which 
learners merely gave vague feedback about whether they liked the course or not. This was 
easier said than done, however, as busy instructors and learners naturally tended to focus on 
their primary goal of "learning" rather than "evaluating" and thus couldn't always give the 
time to evaluation that the evaluators would like. For these non-evaluation-specialists, 
evaluation naturally seemed to be something “ academic " or done to satisfy funders (or keep 
evaluators employed!) rather than something really useful for those doing the work of 
teaching and learning. (This was despite considerable effort by the evaluators to discuss the 
value of evaluation and to customize the evaluation to each site.) 

In addition to a na tural- resist ance-to-evaluation^ it-is under the best of circumstance also 
difficult to trackwhether and how learners are actually using what they learn back at work 
and, if they are, what impact such applications have on complex, ever-changing 
organizations. And what are we to conclude if a consciencious learner does try to use his 
new skills but his productivity goes down afterward? Is the education program a failure? Is 
it fair or accurate to attribute a change (positive or negative) in job performance to a single 
factor like an education program? 

For these reasons it was difficult to systematically document the impacts of the education 
activities on the many learners at each site. There was nonetheless evidence that the 
program did have significant impact on several companies, especially in terms of changing 
their approaches to training and team management. 

One company, for example, reported a year after CLO formally ended that it had 
adapted many of the elements of the CLO model, including: 

0 The company had re-designed all its basic skills activities to build in a fuller range of 
SCANS competencies (rather than just focus on "raising workers' reading scores"). 

0 The company was now using project-based learning (rather than traditional skill-and-drill 
workbooks which had learners develop skills outside of context), with the expectation that 
learners would leave the classes with particular strategies for workplace improvements in 
hand and then try to apply them back on the job. 

0 The company had reconfigured its human resource development strategies to ensure that 
managers encouraged learners to use what they were learning in the education classes and 
then reward them if positive improvements in workplace operations occurred as a result. 

0 To further re-inforce the connection between the basic skills program and the workplace, 
the company was now bringing technical trainers and supervisors into the basic skills 
classroom to serve as resource persons and guest instructors. 

0 The company was taking this experience with its current workforce a step further and 
getting more involved in local school-to-work activities. Company personnel were 
explaining to local school teachers what skills workers needed and the innovative 
educational practices which could be used to help learners learn them. 
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The Central Planning Team 

As explained in previous chapters, the seven-site CLCI project was managed by a Central 
Planning Team (CPT) composed of representatives from the sites, the project director (the 
workplace education coordinator at the New York State Education Department), two 
evaluation consultants, and a curriculum advisor. 

The purposes and make-up of the CPT emerged in the first six months of the project, 
when key members on what became the CPT realized that some kind of collaborative decision- 
making structure would be needed to facilitate communications and planning across a seven-site 
project spread across northern New York State. 

These decision-makers also realized that such a structure would be consistent with the 
team decision-making which was central to what was being proposed at the classroom and site 
levels. These decision-makers agreed that they should themselves practice what they were 
preaching, form what they eventually called the Central Planning Team, and use it as a vehicle 
for a collaborative process of setting goals for the project, ensuring that the sites got the supports 
they need, and monitoring progress and needs at the site level. 

This collaborative structure put greater responsibility and control for the project into the 
hands of site representatives and outside evaluators and consultants. As such, it was a departure 
from more-traditional ways of running statewide workplace education projects. More typically, 
only a small number (sometimes one person) of people set guidelines, distribute funds, and monitor 
program quality for a multi-site government-funded initiative. 

Recognizing the need to make this collaborative process as efficient and responsive as 
possible to needs and opportunities as they arose, the CPT set up several communications 
channels (e.g., periodic meetings, mailings, site visits, and one-to-one phone calls). The project also 
set up an electronic listserv for CPT members, but this was used infrequently, largely because few 
members had Internet access during the life of the project. 

The exact make-up of the CPT varied over the three years, but shown below are people 
who attended CPT meetings fairly regularly. The names of the site coordinators appear regularly 
elsewhere in this text. (Those interested in learning more about CLCI are urged to contact any of 
these resource persons for more information.) 

Over the three years, the CPT met three or four times each year, to give updates on 
what was happening at the sites, what the evaluators and curriculum consultant were doing, and 
funding-related information, as well as to plan staff workshops. The energy of the CPT ebbed 
and flowed over three years, as members had to deal with other projects they were working on. 

Overall, members agreed that a multi-site project can benefit from a well-run 
collaborative decision-making structure, but that such a team requires commitment, time, expertise 
of members and clear goals, timelines, and responsibilities for all involved. 
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Members of the Central Planning Team 



Project Director 
Bob Knower 

Office of Workforce Preparation and Continuing Education 

New York State Education Department 

Education Bldg., Room 305M EB 

Albany, NY 12234 

518/474-3302 

Evaluation Consultants 
Don Cichon 

Donald Cichon Consultants 
40 Ash Street 
Dover, NH 03820-3947 
603/740-9331 



Laura Sperazi 
Evaluation Research 
RR#1, Box 2355 
Newfane, VT 05345 
802/365-7688 



Curriculum Advisor 



Paul Jurmo 
Learning Partnerships 
14 Griffin Street 

East Brunswick, NJ 08816-4806 
732/254-2237 

Albany International/East Greenbush 



Site coordinator 
Margaret Shirk 
152 Hudson Avenue 
Delmar, NY 12054 
518/475-0412 

Hudson Valley Community College representative 

Mike Armstrong 

Corporate Programs, PAC 519 

Hudson Valley Community College 

80 Vandenburgh Avenue 
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Troy, NY 12180-6096 
518/270-4830 



Albonv International representative 

Vincent Partisano 

Albany International Corporation 

Press Fabrics Division 

P.O. Box 1109 

Albany, NY 12201 

518/285-4264 



Albany International/Menands 

Site coordinator 
Judy Lees 

60 Apple T ree Lane 

Clifton Park, NY 12065-2106 

518/270-1549 

Hudson Valiev Community College representative 

Mike Armstrong 

Corporate Programs, PAC 519 

Hudson Valley Community College 

80 Vandenburgh Avenue 

Troy, NY 12180-6096 

518/270-4830 



Albany Intemational/Homer 



Site coordinator 
Paula Hayes 

UAW Chrysler Region 9 
6075 E. Mol by Road 
Syracuse, NY 13211 
315/463-7185 



Albany International representative 
Linda Holland 

Albany International Corporation 
1565 S. Main Street 
Homer, NY 13077 
607/749-7226 
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E.G.&G. Wriaht Components 



Site coordinators 



Dianne Spang 

K-12 Coordinator 

Office of Associate Provost 

Rochester Institute of Technology 

One Lomb Memorial Drive 

Rochester, NY 14623 

716/475-2984 

Bev Zier 

Wayne-Finger Lakes BOCES 
2501 County Road 20 
Stanley, NY 14561 
716/526-4654 



EG&G Wriaht Components representative 

Cynthia Schwab 

EG&G WQright Components 

Route 96, PO Box 160 

Phelps, NY 14532 

315/548-9501 



Eastman Kodak Corporation 



Site coordinators 
Karl Kania and Chris Schweiker 
Family Learning Center 
Rochester City Schools 
30 Hart Street 
Rochester, NY 14605 
716/262-8946 

Eastman Kodak Corporation representative 

Shyrl Orrego Scalice 

Eastman Kodak Corporation 

343 State Street 

Rochester, NY 14650-0811 

716/722-5045 

Delphi Corporation/United Auto Workers 

Site coordinator 
Karl Kania 

Family Learning Center 
Rochester City Schools 
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30 Hart Street 
Rochester, NY 14605 
716/262-8946 



Instructo rs 



Tom Wager 
Family Learning Center 
Rochester City Schools 
30 Hart Street 
Rochester, NY 14605 
716/262-8946 



Gina Porter 
c/o UAW Local 1097 
221 Dewey Avenue 
Rochester, NY 14608 
716/647-7942 



UAW representatives 

Bob Trouskie and Patti Campbell 

UAW Local 1097 

221 Dewey Avenue 

Rochester, NY 14608 

716/647-7413 



Delphi representative 
Chuck Tisa 
Delphi Corporation 
PO Box 1790 
Rochester, NY 14692 
716/617-4973 



Elmira Stamping & Manufacturing Company 



Site coordinator 
Stan Swider 

Schuyler-Chemung-Tioga BOCES 
459 Philo Road 
Elmira, NY 14903 
607/739-3581 
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CHAPTER 8 
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educators . Don Mills, ON: ABC CANADA. Part of ABC CANADA'S series on collaborative 
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literacy . Washington, D.C.: AFL-CIO Human Resources Development Institute.Calls for 
greater worker inyolvement in workplace education programs. 
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and the roles of stakeholders. 

Soifer, R. et al. The complete theory-to-practice handbook of adult literacy: Curriculum design 
and teaching approaches. New York: Teachers College Press, 1990. A summary of how to 
use whole language in adult literacy education. The authors developed this model in auto 
manufacturing plants in the Detroit area. 

Stein, S.G. (May 21, 1993). "Continuous learning for continuous improvement, or basic skills, 
worker empowerment, and high performance work organizations: Why you can't have one 
without the other." Keynote address at May 1993 annual conference of the Texas 
Workforce Education Consortium, Dallas, TX. Calls for new forms of workplace learning 
which more directly integrate learning into day-to-day shop-floor work activities. 

Stein, S.G. and Sperazi, L. (October 15, 1991). "Workplace education in context: A chart 
comparing traditional and high performance work organizations." Part of Tradition and 
change: The role of workplace education in the transformation of the workplace, a paper 
presented by Sondra G. Stein at the annual meeting of the American Association for Adult 
and Continuing Education, in Montreal. Succinctly contrasts features of traditional and high- 
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traditional and high performance workplaces approach goal-setting, curriculum, scheduling, 
assessment, evaluation, and other key components. 

Turk, J. and Unda, J. "So we can make our voices heard: The Ontario Federation of Labour's 
BEST project on worker literacy." In M.C. Taylor, G.R. Lewe, & J.A. Draper (eds.), Basic skills 
for the workplace (pp. 267-280). Toronto: Culture Concepts. Presents an alternative 
perspective on workplace education which features greater worker involvement and a more 
holistic view of what an education program should focus on. 

Workplace Needs Assessment 

Folinsbee, S. and Jurmo, P. Collaborative needs assessment: A handbook for workplace 
development planners . Don Mills, ON: ABC CANADA, 1994. Presents a step-by-step 
process for conducting a workplace needs assessment. 
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Portfolio Assessment and Program Evaluation 

Barker, K.C. (October 1991). A program evaluation handbook for workplace literacy . Ottawa: 
National Literacy Secretariat. An overview of issues and approaches related to workplace 
literacy evaluation. 

Fingeret, H.A. (1993). It belongs to me: A guide to portfolio assessment in adult education 
programs . Durham, NC: Literacy South. A comprehensive guide to the use of portfolio 
assessment for adult literacy programs. Can be adapted for use in workplace programs. 

Jurmo, P. and Folinsbee, S. Collaborative evaluation: A handbook for workplace development 
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and the role evaluation might play . East Brunswick, NJ: Literacy Partnerships. Begins with an 
overview of purposes for and approaches to workplace basic skills education. Then analyzes 
the current state of evaluation in workplace programs. Profiles a number of projects which 
support the creation of a new, "collaborative" approach to workplace education. Concludes 
with recommendations for workplace educators, employers, unions, funders, and learners. 

Kinsey, D.C. (1978). Evaluation in nonformal education: The need for practitioner evaluation . 
Amherst, MA: University of Massachusetts, Center for International Education. Shows why 
adult educators should see evaluation as a useful tool for improving their program rather than 
an external imposition. 

Lewe, G.R. (1993). Literacy task analysis . New Westminster, British Columbia: B.C. Construction 
Industry Improvement Council. Presents an alternative approach to literacy task analysis, 
which more actively involves learners and focuses on identifying skills needed for 
advancement within trades. 

Lytle, S.L. and Wolfe, M. (1989). Adult literacy education: Program evaluation and learner 
assessment . Columbus, OH: ERIC Clearinghouse on Adult, career, and Vocational Education. 
A comprehensive overview of the current state of adult literacy assessment arid evaluation. 
Recommends new ways of looking at these important program functions which emphasize 
greater involvement of learners and other stakeholders in analyzing needs and progress of 
individuals and programs. 

MacKillop J. Whole language for adults: A guide to portfolio assessment . Syracuse: New 
Readers Press, 1994. Describes the rationale and processes for using portfolio assessment to 
evaluate what students accomplish in adult basic skills programs. 

Mikulecky, L. and Lloyd, P. (1993). The impact of workplace literacy programs: A new model for 
evaluating the impact of workplace literacy programs . Philadelphia, PA: National Center on 
Adult Literacy. Summarizes issues and options facing evaluators of workplace education 
programs and presents sample instruments for monitoring program impact. Suggests going 
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beyond focusing solely on changes in decontextualized skills and/or job performance to 
tracking impact on learner beliefs about literacy, self, and education; and improved literacy 
practices at work and elsewhere. 

Philippi, J. W. (May 1992). How do you know if it's working? Evaluating the effectiveness of 
workplace literacy programs . Springfield, VA: Performance Plus Learning Consultants, Inc. 
Summarizes issues and options for evaluators of workplace education programs. These 
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to stakeholder interests, and integrating education with organizational development. 

Sarmiento, A.R. (1993). "Alternative designs for evaluating workplace literacy programs." 
Proceedings of U.S. Department of Education work group conference on "Design Guidance 
for Evaluating Workplace Literacy Programs." Research Triangle Park, NC: Research Triangle 
Institute. Argues that workplace education programs funded by the U.S. Department of 
Education have not been adequately evaluated. Says that program goals need to be more 
clearly articulated and not defined narrowly as improvement of only the literacy skills required 
by participants' current jobs. States that education programs require a supportive context -- 
one committed to transformation to a new way of organizing work -- to produce meaningful, 
lasting results. 

Sperazi, L. and Jurmo, P. (July 1994). Team evaluation: A guide for workplace education 
programs . East Brunswick, NJ: Literacy Partnerships. Shows stakeholders in workplace 
education programs how to plan and carry out a collaborative evaluation. Based on an 18- 
month project funded by the National Institute for Literacy. 

Sperazi, L. and Jurmo, P. (June 1994). Team evaluation: Case studies from seven workplace 
education programs . East Brunswick, NJ: Literacy Partnerships. Describes how collaborative 
evaluation was carried out in seven workplace programs (six in the U.S. and one in Canada) 
and analyzes lessons learned in the seven sites -- strengths, limitations, and needed 
improvements of the team evaluation concept. 

Sticht, T.G. (April 20, 1991). Evaluating national workplace literacy programs . El Cajon, CA: 
Applied Behavioral & Cognitive Sciences, Inc. Summarizes issues and options for evaluators 
of workplace education programs. Encourages stakeholders to see evaluation as a tool for 
stregthening their programs -- in the same "spirit of inquiry" program participants should be 
adopting. 

Taylor, M.C. (1991). " How to approach worker testing and assessment." In M.C. Taylor, G.R. 
Lewe, & J.A. Draper (eds.), Basic skills for the workplace (pp. 183-202). Toronto: Culture 
Concepts. Summarizes job-related goals of workplace education programs, standardized and 
alternative ways of assessing learner progress in literacy programs, and the implications of 
using those assessment tools in workplace settings. Says that, when selecting assessment 
tools, stakeholders need to be clear what the education is program is trying to achieve and 
should involve learners in decisions related to program goals and assessment procedures. 
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High-Performance Organizations 

Sarmiento, T. (July 1991). Do workplace literacy programs promote high skills or low wages? 
Suggestions for future evaluations of workplace literacy programs. Labor Notes, a monthly 
newsletter of the Center for Policy Research of the National Governors Association, 
Washington, D.C. States that a workplace education program have the choice of preparing 
workers for old, "top-down" workplaces or new, "high performance" organizations. In the 
latter, new workplaces, education programs should be structured in the same collaborative 
way in which other workplace activities are carried out. 

Senge, P.M. (1990). The fifth discipline: The art and practice of the learning organization. New 
York: Doubleday/Currency. Makes the case for a new kind of workplace which sees 
ongoing, collaborative learning and decision-making as the thread which weaves the 
organization together. 
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